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cepts  and' results  are  presented  that  apply  to  any  zero-sum  two-person  differential 
game.  The  direct  method  of  Pontryagin,  specifically  dealing  with  linear  pursuit- 
evasion  games,  is  presented  and  discussed.  It  is  shown  how  it  applies  to  several  in¬ 
formation  structures.  An  interesting  question  is  that  of  the  optimality  of  the  strat¬ 
egies  generated.  It  turns  out  to  be  closely  related  to  the  continuous  limit  of  the 
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these  strategies  from  being  glo1  ally  optimal  is  described,  providing  criteria  to  checkl 
this  optimality.  An  analysis  is  given  of  Pshenichnyi 1 s  nonregular  points,  linking 
them  with  the  abnormal  problem  of  the  calculus  of  variations  and  with  Isaacs’  concept 
of  a  barrier.  Pontryagin’ o  technique  is  also  applied  to  multistage  games,  the 

main  emphasis  being  on  a  system-theoretic  formulation  where  the  controls  are  unbounded 
and  the  capture  set  is  a  subspace.  Explicit  criteria  are  given  for  completion  to  be  1 
possible  with  the  three  main  information  structures.  Following  Kalman,  special  atten¬ 
tion  is  given  to  the  case  where  the  capture  subspace  is  a  submodule  of  the  system,  andl 
his  strong  controllability  theorem  is  generalized.  The  second  part  of  this  study 

is  an  investigation  of  a  specific  example;  Isaacs'  Isotropic  Rocket.  The  previous 
technique  is  applied  to  it,  and  readily  gives  interesting  results.  However,  because 
of  the  phenomenon  mentioned  above,  the  corresponding  trajectories,  Isaacs'  primaries, 
are  not  always  optimal.  The  investigation  is  pursued  with  the  more  classical 
Hamilton-Jacobi  theory,  generalized  by  Isaacs  to  a  game-theoretic  fora  of  Bellman  s 
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ABSTRACT 


This  work  is  primarily  a  study  of  linear  pursuit-evasion  games, 

t  although  several  concepts  and  results  are  presented  that  apply  to  any 

zero-sum  two-pei*son  differential  game. 

The  direct  method  of  Pontryagin,  specifically  dealing  with  linear 
pursuit-evasion  games,  is  presented  and  discussed.  It  is  shown  how  it 
applies  to  several  information  structures.  An  interesting  question  is 
that  of  the  optimality  of  the  strategies  generated.  It  turns  out  to 
be  closely  related  to  the  continuous  limit  of  the  discretized  informa¬ 
tion  structure  used,  and  of  the  induced  e-strategies.  It  is  shown  that 
the  limit  strategies  are  locally  optimal.  A  condition  is  also  found 
under  which  there  are  e-strategies  enjoying  the  same  property.  The 
phenomenon  that  can  prevent  these  strategies  from  being  globally  opti¬ 
mal  is  described,  providing  criteria  to  check  this  optimality.  An  an¬ 
alysis  is  given  of  Pshenichnyi 's  nonregular  points,  linking  them  with 
the  abnormal  problem  of  the  calculus  of  variations  and  with  Isaacs' 
concept  of  a  barrier. 

.  Pontryagin's  technique  is  also  applied  to  multistage  games,  the 

main  emphasis  being  on  a  system-theoretic  formulation  where  the  controls 

*  are  unbounded  and  the  capture  set  is  a  subspace.  Explicit  criteria  are 
given  for  completion  to  be  possible  with  the  three  main  information 
structures.  Following  Kalman,  special  attention  is  given  to  the  case 
where  the  capture  subspace  is  a  submodule  of  the  system,  and  his  strong 
controllability  theorem  is  generalized. 

The  second  part  of  this  study  is  an  investigation  of  a  specific 
example;  Isaacs'  Isotropic  Rocket.  The  previous  technique  is  applied 
to  it,  and  readily  gives  interesting  results.  However,  because  of  the 
phenomenon  mentioned  above,  the  corresponding  trajectories,  Isaacs' 
primaries,  are  not  always  optimal.  The  investigation  is  pursued  with 
the  more  classical  Hamilton-Jacobi  theory,  generalized  by  Isaacs  to  a 
game-theoretic  form  of  Bellman's  dynamic  programming. 

The  game  of  kind,  whore  the  payoff  is  capture  or  escape,  is  first 
'  investigated.  This  determines  barriers  that  can  either  define  a  closed 
captui’e  x'egion  or  represent  discontinuities  of  the  optimal  time  to  go. 

* 
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The  concept  of  cone  of  semipermeable  directions  is  emphasized,  and  a 
geometrical  construction  of  it  is  given.  This  concept  is  used  to  pre¬ 
sent  Isaacs'  "envelope  barrier."  It  is  shown  that  for  a  certain  range 
of  parameters  this  barrier  does  not  provide  the  complete  solution  of 
the  game  of  kind.  Two  other  semipermeable  surfaces  are  attached  to  it, 
which  sometimes  succeed  in  defining  a  closed  capture  region.  When  they 
do  not,  two  more  surfaces  are  constructed,  but  they  do  not  close  the 
barrier  either.  Two  new  concepts  are  introduced:  the  "envelope  junc¬ 
tion,  "  which  is  a  way  in  which  two  semipermeable  surfaces  can  join  at 
a  nonzero  angle  and  still  form  a  barrier,  and  the  "singular  surface, " 
which  is  a  semipermeable  surface,  the  trajectories  of  which  all  come 
together  at  a  singular  point. 

Finally,  the  game  of  degree,  where  the  payoff  is  the  time  of  cap¬ 
ture,  is  investigated.  As  was  pointed  out  by  J.  V.  Breakwell,  the  op¬ 
timal  solution  involves  trajectories  having  a  state  constrained  arc. 

The  concept  of  singular  state  constraint  is  introduced.  It  is  shown 
that  nonsingular  constraints  are  reached  and  left  tangentially  by  the 
optimal  trajectories.  The  general  corner  condition  for  differential 
games  is  derived.  It  includes  the  "indifference  condition,"  two  par- 

* 

ticular  cases  of  which  are  Isaacs'  "equivocal  surface"  and  Breakwell 's 
"switch  envelope."  In  the  present  game  it  is  the  latter  form  that  oc-  / 

curs,  but  the  equations  of  the  switching  surface  are  extremely  involved, 
and  numerical  integration  of  them  was  not  feasible  in  this  study.  Some 
analytical  results  are  derived  on  the  shape  of  this  surface,  and  conjec¬ 
tures  are  presented  on  how  the  complete  solution  of  the  game  may  look. 
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INTRODUCTION 


This  work  is  devoted  to  the  study  of  linear  pursuit-evasion  games,  a 
special  class  of  two-person  zero-sum  dynamical  games.  Dynamical  games 
include  differential  games,  where  a  continuous  system  is  governed  by  a 
set  of  differential  equations,  and  multistage  games,  where  a  discrete 
system  is  governed  by  a  set  of  difference  equations. 

Problems  of  pursuit  appeared  in  the  Russian  literature  at  an  early 
stage  of  development  of  modern  control  theory.  A  well-known  example  is 
the  pursuit  problem  of  Kelendzeridze  in  the  book,  Ma thematical  Theory  of 
Optimal  Processes ,  by  Pontryagin,  e_t  al_.  But  the  full  concept  of  the  dif¬ 
ferential  game  was  first  introduced  by  Rufus  Isaacs  in  various  Rand  re¬ 
ports  as  early  as  1954,  and  in  his  book,  Differential  Games ,  in  1965. 

The  distinctive  feature  is  that  both  players  try  to  do  the  best 
possible  with  no  a  priori  knowledge  of  what  the  opponent  is  going  to  do. 

A  striking  difference  with  a  two-sided  control  problem,  where  one  player 
knows  the  whole  future  control  of  the  other  one,  appears  in  the  fact  that 
while  Kelendzeridze  was  able  to  apply  the  maximum  principle  to  the  latter 
case,  Pontryagin  (Reference  [21])  investigated  the  use  of  this  technique 
for  the  former  case  and  found  that  it  usually  does  not  apply. 

In  the  zero-sum  game,  the  only  one  we  shall  consider,  there  is  a 
performance  index  which  one  player,  whom  we  shall  call  P  ,  for  pursuer, 
tries  to  minimize,  and  the  other  player,  whom  we  shall  call  E  ,  for 
evader,  tries  to  maximize.  Therefore,  we  have  a  minimax  problem  very 
much  like  the  corresponding  case  of  the  classical,  static,  game  theory  of 
Von  Neuman  and  Morgenstern.  But  here  the  game  has  dynamics,  and  the 
strategies  sought  are  closed-loop  control  laws. 

A  very  important  question  in  such  games  is  that  of  the  information 
structure.  At  present,  the  game  theoretic  form  of  the  most  basic  con¬ 
cepts  and  tools  to  handle  partial  information— observability,  filtering 
techniques — is  missing  or  unsolved.  We  shall  consider  only  detex-ministic 
structures.  It  will  always  be  assumed  that  both  players  know  the  state 
perfectly.  The  information  available  to  them  on  each  other's  control 
will,  however,  be  varied. 
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In  a  pursuit-evasion  game,  the  game  is  "completed"  when  the  state 
enters  a  capture  set.  The  performance  index  is  the  time  the  game  lasts 
until  completion,  or  "capture."  The  evader  tries  to  survive  as  long  as 
he  can  while  the  pursuer,  on  the  contrary,  strives  to  capture  him  as 
quickly  as  possible.  This  type  of  game  has  provided  several  of  Isaacs' 
problems.  One  of  these  will  be  considered  here,  in  the  second  part. 

In  the  first  three  chapters,  we  discuss  a  direct  method  of  Pontryagin, 
specifically  dealing  with  linear  pursuit  evasion  games. 

In  the  first  chapter,  this  method  is  introduced  and  discussed.  It 
is  extended  slightly  to  allow  various  information  structures,  differing 
by  the  amount  of  information  available  to  the  pursuer  on  the  evader's 
future  control,  including  the  case  where  this  whole  future  is  known.  The 
object  of  this  technique  is  to  provide  an  "estimating  function"  T(z) 
such  that  for  a  game  starting  at  z  ,  capture  is  surely  possible  in  a 
time  no  longer  than  T(z)  . 

In  the  second  chapter,  our  main  objective  is  to  study  the  optimality 
of  the  process  proposed  by  Pontryagin.  We  use,  following  Fleming  and 
Friedman,  e-strategies,  consistent  with  the  information  structure  intro¬ 
duced  in  the  first  chapter.  We  find  a  condition  under  which  our  estimat¬ 
ing  function  can  be  optimal  with  such  strategies,  for  small  enough  e  . 
Then  it  is  seen  that,  without  this  condition,,  T(z)  can  still  be  optimum 
for  the  continuous  process,  e-strategies  actually  yielding  capture  times 
arbitrarily  close  to  it  as  e  is  decreased.  The  limit  of  the  e-strate¬ 
gies  is  investigated  and  characterized.  It  is  shown,  then,  how  the 
estimating  function  and  the  corresponding  strategies  can  still  fail  to  be 
optimal,  essentially  because  they  can  lead  to  trajectories  that  would 
Me  inside  the  capture  set  for  some  time  before  the  calculated  capture 
instant.  A  necessary  and  sufficient  condition  for  this  not  to  happen  is 
discussed.  Finally,  non-regular  points  are  briefly  investigated,  and  one 
kind  identified  with  Isaacs'  barriers. 

The  third  chapter  is  the  only  one  dealing  with  multistage  games, 
e-strategies  naturally  lead  to  a  discrete  version  of  the  game  which  is 
briefly  considered,  mainly  from  the  point  of  view  of  the  information 


structure.  This  yields  the  concepts  necessary  for  applying  the  previous 
techniques  to  the  system  theoretic  formulation  of  the  multistage  game, 
with  unbounded  controls.  In  that  case,  the  estimating  function  is  shown 
to  be  optimal.  The  particular  case  where  the  capture  subspace  is  a  sub- 
module  is  investigated  and  an  earlier  result  by  Kalman  [19]  generalized. 

The  remaining  three  chapters  deal  with  a  specific  example:  Isaacs' 
Isotropic  Rocket  Game,  described  in  [17]  and  [18], 

In  the  fourth  chapter,  this  game  is  introduced,  and  its  various 
descriptions  presented.  Then  we  apply  to  it  the  method  of  the  first 
chapter,  which  rapidly  yields  an  estimating  function.  However,  this  es¬ 
timating  function  is  not  optimal  in  the  whole  state  space,  the  condition 
of  Chapter  Two  being  violated. 

In  the  fifth  chapter,  we  use  Isaacs'  technique  and  try  to  solve  the 
"game  of  kind"  qualitative  game,  the  outcome  of  which  is  capture  or 
escape.  It  turns  out  that  Isaacs'  conjectures,  according  to  which  his 
solution  would  have  been  complete,  are  not  verified  for  all  values  of 
some  parameters.  Trying  to  complete  this  solution  leads  to  the  concept 
of  envelope  junction,  a  corner  condition  for  barriers.  A  new  type  of 
barrier  is  also  introduced,  the  singular  barrier,  where  all  member  tra¬ 
jectories  meet  at  a  singular  point.  But  we  have  not  been  able  to  finish 
the  problem  completely,  due  to  the  fact  that  the  solution  seems  to  be 
linked  to  another  unsolved  problem,  of  the  following  chapter. 

The  sixth  and  last  chapter  deals  with  the  "game  of  degree"  quantita¬ 
tive  game,  the  outcome  of  which  is  longer  or  shorter  capture  time.  It  is 
shown  that  the  solution  involves  a  "safe  contact,"  first  perceived  by 
J.  V.  Breakwell  and  already' investigated  under  his  supervision.  The  oc¬ 
currence  of  this  safe  contact  is  iinked  to  the  phenomenon  pointed  out  in 
Chapter  Two,  causing  the  estimating  function  to  be  non-optimal.  Then,  we 
need  to  allow  a  corner  in  the  optimal  trajectories.  The  general  corner 
condition  for  differential  games  with  integral  payoff  is  derived,  largely 
using  the  concept  of  field,  and  resting  upon  ideas  developed  by  Isaacs 
and  Breakwell.  However,  the  partial  differential  equation  it  leads  to  in 
this  game  is  so  complicated  that  we  were  unable  to  integrate  it  numeri- 
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cally.  Some  analytical  results  are  derived  concerning  the  qualitative 
shape  of  its  solution,  and  conjectures  presented  on  how  the  solution  of 
the  game  may  look. 


1.  THE  DIRECT  METHOD  OF  PONTRYAGIN 

In  this  chapter,  we  present  the  method  developed  by  Pontryagin  [22, 
23,24]  to  deal  with  linear  pursuit-evasion  games.  In  doing  so,  we  dis¬ 
cuss  how  the  method  can  apply  to  various  information  structures. 

1 .1  Statement  of  the  Problem 

In  an  n-dimensional  Euclidean  vector  space  E  ,  a  system  is  governed 
by  the  differential  equation 

£|«Cz-u  +  v  (1.1) 

where 

z  is  the  state  of  the  system  z  £  E  ; 

C  is  a  constant  nXn  matrix; 

u  £  P  is  a  control  variable  chosen  by  the  pursuer; 

v  £  Q  is  a  control  variable  chosen  by  the  evader; 

P  and  Q  are  closed  convex  subsets  of  E  . 

Admissible  control  functions  u(0  and  v(.)  are  measurable  functions 
of  time,  taking  their  values  in  P  and  Q  respectively, 

A  given  subspace  L  of  E  is  called  the  geometrical  space.  The 
orthogonal  projection  of  E  onto  L  is  called  jt  . 

A  given  closed  convex  subset  e  of  L  is  called  the  capture  set. 
The  dimension  of  C  can  be  n  .  In  that  case,  L  =  E  ,  and  the  operator 
is  simply  the  identity. 

Capture  is  defined  as 

n  z  £  G  . 

The  general  problem  can  be  stated  ns  deciding  whether  it  is  possible, 
knowing  the  state  z  at  each  instant,  and  with  some  suitable  information 
on  the  evader's  control,  to  construct  a  control  function  u(0  such  that 
capture  will  eventually  occur.  II  this  is  possible,  in  what  time,  and 
how  should  the  control  u  be  chosen? 
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The  question  of  the  evader’s  "best"  behavior  will  be  considered 
later. 

1 .2  Remarks 

Before  going  into  the  analysis  of  this  problem,  we  shall  make  some 
remarks  about  i.ts  formulation. 

i)  Dynamics .  The  dynamics  described  in  (1.1)  may  seem  somewhat 
restrictive  as  compared  to  the  more  general  formulation 

z  =  C  z  -  Gu'  +  Jv'  .  (1.1a) 

However,  we  restrict  u1  and  v'  to  belong  to  compact  convex  subsets. 
Therefore,  (1.1a)  is  equivalent  to  (1.1)  by  letting 

Gu'  =  u  Jv'  =  v 

and  it  is  straightforward  to  see  that  if  u'  and  v'  belong  to  P'  and 
Q'  ,  closed  and  convex  in  their  respective  spaces,  u  and  v  belong  to 
closed  convex  subsets  P  and  Q  of  E  . 

The  use  of  constant  sets  P  and  Q  corresponds  to  constant  matrices 
G  and  J  ,  which  is  consistent,  with  the  fact  that  we  take  a  constant 
matrix  C  . 

ii)  Many-Control  System.  Another  generalization  one  might  want  to 
consider  is  a  many-control  system: 

z=Cz+u.  +u_+  ...  +u 

12  p 

u,  €  P.  . 

1  i 

But  the  approach  we  take  is  essentially  unsymmetric.  We  investigate  what 
can  be  done  with  the  control  u  ,  knowing  how  the  other  control  can  act 
on  the  system.  Whether  v  is  under  the  control  of  a  single  player  or  is 
the  added  effect  of  several  players'  controls  makes  no  difference. 

We  can  reduce  this  formulation  to  the  first  one,  letting 

-u  «  u  €  P  =  -Pj 

u  +  ...  +  u  =  v  £  Q  =  P  +  ...  +  P  • 

2  p  2  q 
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And  notice  that  Q  is  still  convex. 

iii)  Convexity.  We  insist  on  the  compactness  and  convexity  of  the 
sets  P  and  Q  because  the  following  theory,  at  several  points,  depends 
critically  on  it.  It  is,  therefore,  interesting  to  notice  that,  in  as¬ 
suming  the  convexity  of  these  sets,  there  is  no  loss  of  generality.  It 
is  a  well-known  fact  in  control  theory  that,  for  a  dynamical  system,  any 
-point  of  the  convex  hull  of  a  non-convex  control  set  could  be  approximated 
as  closely  as  desired  by  chattering  between  n  values  belonging  to  the 
control  set  itself. 

The  convexity  of  the  capture  set  e  is  needed  as  well,  and  must  be 
regarded  as  a  restrictive  assumption.  Notice  that  it  is  verified  by  the 
interesting  special  case 

e  =  {0} 

which  corresponds  to  capture  being  defined  as 

z  €  M  L  ©  M  =  E  M  orthogonal  complement  to  L 

1.3  The  Information  Structure 

Following  Pontryagin,  we  introduce  a  special  information  structure, 
which  we  shall  call  the  lower  rule  e  . 

At  each  instant  t  the  pursuer  knows  the  state  z(t)  and  the 
evader's  control  history  v(t)  for  a  time  e  in  the  future,  namely 

v(s)  t  <  s  <  t  +  e  written  v[t,t+e]  . 

A  pursuit  strategy,  then,  is  a  mapping 

[t  ,t  +e] 

u  :  [t  ,t  +e]  X  E  x  Q  °  °  -»  P 

o  o 

written  as 

u  =  u(t;z(t  ),v[t  ,t  +e]\  t  <  t  <  t  +  €  . 

In  this  chapter,  this  definition  will  be  enough  for  our  purposes. 

As  the  system  is  time-invariant,  we  can  arbitrarily  set  t  =  0  . 
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The  question  of  the  updating  of  the  function  u  as  time  goes  on  will  be 
discussed  in  the  second  chapter  (Section  2.3). 

Three  cases  will  be  considered: 

(X)  e  as  big  as  desired  by  the  pursuer.  The  evader's  control  is 
known  by  the  pursuer  for  the  whole  future. 

This  reduces  the  problem  to  a  classical  formulation,  used  by  Kelendzeridze, 
Varaiya  [27],  Kalman  [19],  and  other  authors.  Does  there  exist,  for 
every  control  history  v(«)  a  corresponding  cohtrol  history  u(»)  such 
that  capture  occurs  in  finite  time? 

p)  e  is  a  given  positive  number,  possibly  very  small.  This  is  the 
case  considered  by  Pontryagin.  He  gives,  in  addition,  a  time  of 
capture  valid  for  every  positive  e  .  This  suggests  the  third 
case: 

Y)  e  =  0  :  Only  the  present  value  of  the  evader's  control  is  known. 
This  is  again  a  classical  problem,  considered  by  Pontryagin 
himself  in  [21,24],  and  other  authors.  This  knowledge  has  been 
found,  in  some  cases,  to  be  necessary  for  optimal  strategies  to 
exist. 

We  shall  consider  this  case  as  a  limit  of  the  previous  one,  following 
Fleming  [11,12,13]  and  Friedman  [14,15]. 

1.4  A  Sufficient  Condition 

We  want  to  find  a  sufficient  condition  for  capture  to  be  possible, 
and  an  estimating  function 

T  :  E  ->  R 

such  that  a  state  z  can  surely  be  captured  in  a  time  no  longer  than 
T(z)  .  Pontryagin  reaches  this  objective  by  constructing  a  set  of  cap- 
turable  points  ,  a  function  of  the  real  variable  T  ,  such  that 

•  it  V  =  C 
o 
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•if  zq  £  ,  there  exists  a  pursuit  control  such  that,  with 

o 

z(0)  =  zq  ,  the  solution  of  (1.1)  verifies 
z(e)  £  V 

t  -e 


As  a  consequence  of  the  second  property,  at  time  e  there  exists  a  new 
pursuit  control  insuring  the  same  inclusion  at  time  2e  and  so  on.  Thus, 
eventually 

z(t)€V  or  it  z  (t  )  €  C  . 

o  o  o 

Therefore,  any  mapping  T(.)  of  the  state  space  into  the  reals  such  that 


is  an  estimating  function. 

From  now  on,  the  estimating  function  we  consider  shall  always  be  the 
smallest  real  number  T(z)  such  that  the  above  inclusion  is  verified. 


The  construction  of  V 


vv c  w 


can  depend  on  e  ,  We  must  clearly  have 
V  £1  <  £2  '  V  T 


since  the  control  u  constructed  with  the  rule  e.^  can  also  be  con¬ 
structed  with  the  rule  .  We  shall  also  see  a  construction,  proposed 
by  Pontryagin,  giving  a  set  valid  for  every  positive  e  . 

We  shall  actually  find  a  family  of  sets  W^  in  L  ,  and  define 
as 


V  =  <t>(-T)jT1W  =  {z|*<5>(t)z  €  W  } 

T  T  T 

tC 

where  <Mt)  =  e  is  the  transition  matrix  associated  with  C  .  We  shall 
have 


W  =  C 
o 


V  =  (z|jt  z  £  C} 


so  that  the  family  V  verifies  the  first  condition  it  V  =  C  . 
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1 . 5  Properties  of  the  Sets  Wy 

It  is  straightforward  to  verify  that  the  solutions  of  (1,1)  verify 

fT 

-t)z(t)  =  )z(0)  +  rt  \  °  <J>(r)[v(T  -r) 

o  o  1  ,  o 

Jr  -t 
o 


-  u(t  -r)  ]dr  . 


(1.2) 


To  simplify  the  equations,  we  introduce  the  notations 


u  =  n<l>(r)u 
r 


P  =  n4>(  r)P 
r 


v  =  rt<I>(r)v 
r 


Q  =  ird>(r)Q  . 
r 


Now,  assume  that  we  have  the  inclusion 


*4>(t  )z  (0)  €  W 

O  X 


(1.3) 


Given  v[0,€]  we  want  to  be  able  to  construct  a  control  u[0,e]  such 
that 


if< I>(t  )z  (0)  + 
o 


f  0  vr(x-r)dr  -  f  °  u^(x- 

J  x  -  e  r  J x  -e  r 


r)dr  €  W 

'  x-e 


'o  '  0 

Or,  defining  the  sum  of  a  set  and  a  vector  in  the  usual  way 


Jx 

°  v  (x-r)dr  e  w  + 
r  x-e 

x-e 

o  o 


/•X 

[  u 

Jx-e  1 


(x-r)dr  . 


Define  the  integral  of  a  set  P^  as  the  set  of  all  possible  integrals  of 

functions  u  taking  their  values  in  P  .  The  existence  of  a  function 
r  •  r 

u  verifying  the  last  inclusion  is  equivalent  to 


«<I>(x  )z(0)  + 
o 


Jr  -i 


v  (x-r)dr  £  W  + 
r  x-e 


f; 


P  dr  . 
r 


This  must  be  true  for  every  possible  control  v(0  ,  so  this  is  equivalent 
to 
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c T 

«$(t  )z(0)  +  1  °  Q 
o  1  r 

J  i  -e 
o 

Finally,  as  we  want  this  to  be  true  for  every  point  verifying  (1.3),  this, 
in  turn,  is  equivalent  to 

51  />  T 

°  QdrCw  +1°  Pdr,  (1.4) 

r  T  -e  1  r 

x  -e  o  J  t  -e 

o  o 

which  is  the  characteristic  property  of  the  sets  W  . 

T 

1.6  Geometric  Subtraction 

In  order  to  ease  the  handling  of  relation  (1.4),  Pontryagin  intro¬ 
duces  the  following  operation: 

Given  two  subsets  A  and  B  of  a  vectorial  space,  define  their 
geometric  difference  as 

D  =  A  i  B  D  =  {z|b  +  z  Q  A} 

which  means  that  D  is  the  biggest  set  such  that 

D  +  B  £  A  . 

If  D  is  non-empty,  we  say  that  A  star  includes  B  :  AS  B  ; 

we  say  that  the  property  of  complete  sweeping  is  verified  if:  A  =  B  +  D  . 

Proposition: 

i)  if  A  and  B  are  convex,  their  geometric  difference  is 
convex; 

ii)  A  *  (B+C)  =  (A-B)  *C=A*B*C=A*C*B; 

iii)  (A+B)  *  C  2  (A-C)  +  B  . 

Proof : 

i)  Let  D  =  A  *  B  be  non-empty.  Let  dx  and  dg  belong  to 
D  .  By  definition,  for  every  b  €  B  ,  their  exists  an 
a^ (b)  €  A  such  that 

di  +  b  +  ai^  »  i  =  1,2  ' 


dr  £  w 


1  -e 
o 


e t 
1 

•f  i  -( 


P  dr 
r 


(1.3a) 
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Consider  d  =  ad  +  (l-a)d  and  check  that  a (b)  =  Oa  (b) 

X  M  X 

+  (l-a)a2(b)  verifies 

d  +  b  =  a(d  +b)  +- (1-a)  (d- +b)  =-a(b)  .  Vb  - 

if  A  is  convex,  a (b)  £  A  and  the  first  result  is  proved. 
Notice  that  we  do  not  need  the  convexity  of  B  .  In  the 
sequel,  however,  only  convex  sets  will  be  met.  There  is  con¬ 
sequently  no  point  in  stating  the  result  with  more  generality. 

ii)  Consider  A  =  (A-B)  -  C  ,  and  d  £  D  ,  we  have,  by  definition, 

d  £  (A*B)  -  C  <*=>  d  +  C  £  (A-B)  <*=>  (d+C)  +  bCa  . 

Now,  the  addition  (d+C)  +  B  is  associative;  thus,  the  last 
relation  is  equivalent  to 

d  +  (C+B)  C  a  <==>  d  £  A  -  (C+B)  , 

which  proves  the  second  result. 

iii)  Consider  D  =  (A*C)  +  B  ,  and  d  £  D  .  By  definition,  there 
exists  e  £  A  *  C  and  b  £  B  such  that  d  =  e  +  b  .  Now  we 
have 

e  +  CCA=>d  +  CCA  +  b£A  +  B=>d£  (A+B)  *  C 

which  proves  the  third  result.  Notice  that  it  is  generally 
not  true  that  (A+B)  s  C  =  (ASC)  +  B  .  It  is  enough,  to  see 
it,  to  take  B  =  C  and  a  case  where  A  -  C  does  not  have 
complete  sweeping  (see  Fig.  1).  However,  we  have  the  follow¬ 
ing  simple  but  interesting  property: 

Proposition: 

If  A  -  C  has  the  property  of  complete  sweeping,  then  this  is  true 
for  (A+B)  i  C  and  (A+B)  -  C  =  (A*C)  +  B  . 

Proof: 

Let  D  =  (A-C)  +  B  . 

Then  D  +  C  =  (A*C)  +C+B=A+B, 

the  last  equality  because  of  the  assumption  of  complete  sweeping  on 
A  -  C  .  And  this  proves  the  claim. 
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1.7  Construction  of  the  Sets  W 
- ______  x 

Relation  (1.4)  can  be  written  as 

dr  *  (  Q  dr  . 
r  )  r 


W  C  w 


t-e 


f 

J  x-e 


(1.5) 


Depending  on  the  information  structure,  we  can  use  (1.5)  in  different 
ways.  We  consider  the  three  cases  of  Section  (1.3): 

Ct)  €  arbitrarily  large.  We  can  use  e  =  t  ,  and  find  immediately 

(  co) 

the  set  we  call  Wv 

T 


"  Prdr)  *\  Vr  • 
o  J  J  o 

However,  it  is  easy  to  see  that  this  family  will  generally  not 
verify  (1.4)  for  a  smaller  e  .  In  fact,  we  have,  using  the 
first  proposition  of  Section  1.6, 


where  the  right  hand  side  is  the  same  as  in  (1.5).  However, 
because  in  general  the  addition  and  geometric  subtraction  of 


r 

J  T-( 


'  X-€ 


P  dr  and 
r 


Q  dr  do  not  commute,  the  inclusion  is 
r 


strict  and  (1.4)  violated. 


This  set  is  nevertheless  very  important,  because  only  when  it  does 
verify  (1.4)  is  an  e-strategy  optimal.  We  shall  discuss  this  point  in 
Chapter  2. 


0)  e  given  positive.  We  use  the  induction  argument  of  Section  1.4, 
and  construct  the  corresponding  set  W^ 


ne  =  t  . 

This  construction  is  called  by  Pontryagin  the  alternating  sum 
of  the  sets 


ke 


(k-l)e 

(e) 


P  dr 
r 


and 


«ke 


(k-l)e 


Q  dr 
r 


Our  set  W  w  is  defined  only  for  discrete  values  of  t 
t 

T(z)  ,  thus,  takes  only  discrete  values,  but  this  is  not  in 
contradiction  with  our  theory  so  far. 


For  the  continuous  case,  we  will  use  another,  continuous,  definition, 
but  this  one  will  be  used  for  the  case  of  discrete  systems  in  Chapter  3. 

y)  e  goes  to  zero.  We  define  a  set  as  a  limit  of  the  pre¬ 

vious  construction.  The  precise  topology  involved  is  discussed 
in  [23]  and  [24];  it  will  be  presented  in  Section  2.6.  For  the 
time  being,  we  shall  consider  it  as  a  pointwise  limit:  every 
point  belongs  to  it  which  can  be  approximated  as  closely  as 
desired  by  a  point  of  a  .  In  other  words, 

i 


(5  ) 

3ze  e  w 

5i  T 


Z  -  z„ 


It  clearly  gives  the  compact  set.  This  set  is  called  by  Pon¬ 
tryagin  the  alternating  integral  and  denoted  by 


Notice  that  this  is  a  mere  notation.  P  -  Q  may  not  exist, 
f  1  r  r 

and  w''0  still  be  non-empty.  Again  because  of  the  properties 
T 
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verifies  (1.4)  as 


(ft) 

of  the  geometric  difference,  the  set  Wvo' 

/  \  T 

soon  as  6  <  e  .  Thus  W^°  verifies  it  for  every  e  . 

5)  e  =  0  :  another  definition.  Finally,  in  the  case  where  P^ 
for  every  r  ,  we  can  consider  another  set  family 

W*  =  C  +  f  (P  *Q  )dr 
T  J  r  r 

Jo 

and  verify  directly  that  it  has  the  required  property  (1.4): 


Notice  the  particular  feature  of  this  construction:  the  exis¬ 
tence  of  the  corresponding  family  of  sets  depends  only  on 

the  existence  of  P  *  Q  ,  Therefore,  if  for  some  capture  set 

r  r 

C  this  family  exists,  this  will  be  true  with  every  capture  set, 
including  C  =  (0}  ,  or  capture  defined  as  point  coincidence. 

Besides  the  point  we  just  mentioned,  the  use  of  this  definition  will 
be  interesting  in  the  following  discussion. 

1.8  Relative  Size  of  the  W's 
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w 


(V 


2  w 


(6,) 


5?  v  -*  f! 


Q  dr  . 
r 


<V 

If  e2  =  2s^  ,  the  last  expression  is  .  This  argument  used  re- 

2 

cursively  shows  that,  when  they  are  both  defined, 


<0  (€,) 
w  ^  2  w 


T  7 

and  thus,  in  the  limit  as  e  goes  to  zero 


2 


Using  the  same  calculation  to  compare  W 
it  is  seen  that 


(°o)  .  w(.e)  . 

to  W  where  €  =  T/n  , 
T  7 


w(oo)  2w(6) 

7  7 

Finally,  we  compare  W'  to  any  ,  Using  recursively  the  following 

7  7 

calculation: 


W’  =  W'  + 
t  7-e 


r 

•'t-e 


(P  2q  )dr£W  +  f  f  P  dr  5  ( 

r  r  t-€  )  r  l 

J-t-e  Jt-t 


Q  d 
r 


Q  [  W  +  j  P^drj  -  j  Q_dr 


T-€  I  r 
1 7“£ 


T-e 


for  7  =  e  ,  then  2e  ,  and  so  on,  shows  that 


V  C  w(e)  V  e  =  -  n 


7  7 


€  K  , 


and  thus,  taking  into  account  the  other  three  inclusions  derived: 
W'  Q  W(o)  c  /  1  C  W  2  c  w(oo)  . 

7  7  7  7  7 


If,  now,  we  remember  the  definition  proposed  for  T(z)  ,  each  of  these 
sets  gives  a  corresponding  estimating  function,  and  the  above  inclusions 
translate  into 

f  \  (€.. )  (£„)  r  \ 

T' (z)  >TW(z)  >  T  (z)  >  T  (z)  >  (z) 
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where 


T(z)  =  »  if  n<t>(T)z  <L  W  Yx  . 

T 

The  last  three  inequalities  simply  say  that  the  time  we  know  to  be  suf¬ 
ficient  to  capture  increases  as  the  amount  of  information  available  on 
the  evader's  control  decreases.  This  agrees  with  what  we  said  in  Section 
1.4. 

An  interesting  simplification  occurs  when  P  £  Q  for  every  r  , 

r  r 

with  complete  sweeping.  Then.  W*  exists,  and  we  have 

T 

W'  +  (  Q  dr  =  C  +  (  [(P  *Q  )  +  Q  ]dr  *  C  +  f  P^dr 

T  Jo  r  Jo  r  Jo  r 

and  thus 

W;  =  (c  +  ^  Prdrj  i  ^  Qrdr  = 

which,  in  view  of  the  chain  of  inclusions  derived,  proves  that  the  four 
constructions  give  the  same  set  ,  and  we  can  use  the  most  convenient 
construction,  ,  for  instance. 

In  consequence  also,  the  function  T(z)  does  not  depend  on  the  in¬ 
formation  structure.  Assuming  more  knowledge  of  the  evader's  strategy 
does  not  allow  us  to  improve  our  a  priori  estimate  of  the  capture  time. 
However,  once  v(0  is  actually  known,  we  might  be  able  to  take  better 
advantage  of  it  and  capture  in  a  shorter  time. 
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2.  OPTIMALITY 


In  this  chapter,  we  address  ourselves  to  two  problems  which  are  to  a 
great  extent  interconnected:  the  question  of  the  optimality  of  the  pro¬ 
cess  we  describe,  and  the  question  of  its  limit  as  e  goes  to  zero,  what 
we  shall  call  the  limit  process.  We  shall  also  be  obliged  to  introduce 
the  motion  of  regular  point,  and  we  shall  make  a  brief  analysis  of  non¬ 
regular  points. 

2.1  The  Concepts  of  Optimality 

The  approach  taken  so  far  is  essentially  unsymmetric.  We  have  as¬ 
sumed  that  the  pursuer  has  an  information  advantage  over  the  evader. 

This  allows  us  to  ask  for  a  strong  type  of  optimality:  we  want  to  use 
optimally  the  information  available.  We  seek  a  function 

providing  the  minimum  capture  time  over  all  such  functions  for  every 
admissible  history  v[to,tQ+e]  .  Let  J(u,v)  be  the  actual  capture 
time 

J(u°,v)  =  min  J(u,v)  . 
u 

Now,  if  the  evader  plays  optimally,  he  will  choose  his  control  in  such  a 
way  as  to  maximize  the  above  functional,  so  that 

J(u°,v°)  =  max  min  J(u,v)  . 
v  u 

In  the  classical  saddle  point  formulation,  one  only  seeks  a  control  op¬ 
timal  against  the  opponent's  saddle  point  control.  Here,  we  want  to  know 
how  to  modify  the  pursuer's  control  to  take  into  advantage  a  possible 
deviation  of  the  evader  from  his  optimal  control,  and  we  do  not  require 
that  there  exist  a  saddle  point,  although  we  shall  see  further  in  what 
sense  there  is  one. 

It  is  interesting  at  this  point  to  review  the  exact  relation  between 
the  maximin  operation  and  the  saddle  point. 
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Let  J(u,v)  be  a  continuous  functional: 


J  :  PXQ-tll 

where  P  and  Q  are  compact  subsets  of  topological  spaces  A  and  B  . 
Then  J  reaches  its  extrema  on  P  X  Q  . 

Let 

Arg  min  J(u,v)  =  u°(v)  ,  Arg  max  J(u,v)  =  v°(u)  . 
u  €  P  v  £  Q 

Proposition:  The  existence  of  a  saddle  point: 

J(u*,v)  <  J(u*,v*)  <  J(u,v*)  V(u,v)  £  P  X  Q  (2.1) 

is  equivalent  to 

min  max  J(u,v)  =  max  min  J(u,v)  =  J(u*,v*) 
u  €  P  v  €  Q  v  £  Q  u  £  P 

(2.1a) 

u°(v*)  =  u*  v°(u*)  =  V* 

o  o 

Proof :  (2.1a)  implies  (2.1)  trivially,  by  the  definition  of  u  and  v 

Let  us  prove  that  (2.1)  implies  (2.1a): 

by  definition  of  v°  , 

j(u*,v°(u*))  >  J(u*,v*) 

and  by  (2.1) 

J^u*,v°(u*)j  <J(u*,v*)  . 

Thus 

j(u*.v°(u*>)  =  J(u*,v+)  v°(u*) 

where  this  definition  of  v°  is  valid,  if  not  necessarily  unique. 

let  u  provide  the  minimnx: 
o 

J (uq i v° (Uq))  <  j(u,v°(u))  V  u  £  P  . 

Apply  this  with  u  =  u*  ,  and  use  (2.2): 

J(uV*)  sjjuV°(u*))  >  J(Uo*V°(U0))  • 


(2.2) 

Now, 
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By  definition  of  v° 

j(u  ,v°(u  ))  >  J(u  ,  v*) 

\  o’  o  /  -  o’ 

by  (2.1) 

J(u  ,v*)  >  J(u*,v*)  . 
o  — 

Therefore,  all  three  inequalities  reduce  to  equalities,  proving  that 
min  max  J(u,v)  =  J(u*,v*)  v°(u*)  =  v*  . 

U  V 

Clearly,  the  proof  can  be  repeated  mutis  mutandis  to  derive  the  rest  of 
(2.1a),  proving  the  proposition.  If  the  mapping  J  is  injective  (one 
to  one),  then  we  also  have  u  =  u*  ,  v&  =  v*  uniquely. 

The  technique  we  shall  use  to  construct  the  control  u  will  be  to 
have  the  estimating  function  decrease  as  fast  as  possible.  However,  this 
does  not  guarantee  optimality  with  what  we  know  of  the  estimating  func¬ 
tion.  Actually,  defined  as  a  sufficient  time  for  capture,  it  is  not  even 
necessarily  unique.  We  have  no  proof  that  another  f(z)  could  not  be 
found,  that  would  lead  to  a  different  strategy. 

But  assume  now  that  at  each  instant,  E  can  insure  that  capture 
will  not  occur  in  a  time  less  than  T(z)  .  Then,  having  this  function 
decrease  as  fast  as  possible  will  indeed  be  the  optimal  behavior  for  P  . 
T(z)  will  then  be  said  to  be  optimal. 

Therefore,  we  are  led  to  the  investigation  of  the  maximin  strategies, 
and  of  the  optimality  of  the  sufficient  capture  time  we  have  displayed. 

2 .2  A  Result  by  Gusyatnikov  and  Nikolsky 

In  [l6 ],  Gusyatnikov  and  Nikolsky  give  a  sufficient  condition  for 
T(z)  as  defined  here  to  be  optimal,  by  displaying  a  5-efficient  strategy 
for  any  positive  5  ,  for  the  evader.  This  condition  is  the  following: 

Condition  A: 

•  P  Q  Vt  ,  complete  sweeping, 

T  T 

•  V  u  €  p  ,  3  V  6  Q  :  u  -  v  £  P  -Q  Vt  .  (2.3) 

T  T  T  T 


21 


Condition  A  says  that  the  element  v  which  provides  the  complement  v 

T 

to  u  in  P  *  Q  is  independent  of  x  .  In  particular,  if  for  some 
XXX 

x  ,  v  is  unique,  then  this  v  will  verify  (2.3)  for  every  x  . 

Under  this  condition,  a  5-efficient  strategy  is  displayed,  of  the 

form 


v°' =  v  ^t,z(tQ)  ,  u[to-e,to]j 

in  terms  of  the  constant  complement  of  u  ,  And  for  every  positive  5  , 
there  is  an  e  small  enough  so  that  this  strategy  insures  that  capture 
will  not  occur  in  a  time  less  than  T^z(0)j  -  5  .  (This  is  the  defini¬ 
tion  of  a  5-efficient  strategy.) 

We  see  then  that  it  should  be  possible  to  define  properly  the  limit 
of  this  strategy,  which  would  yield  the  capture  time  T^z(0)j  .  Actually, 
we  shall  try  to  display  an  optimal  strategy  (O-efficient)  with  a  finite 
e  ,  the  limit  of  which  as  e  goes  to  zero  will  be  obvious. 


2 . 3  Jumps  of  the  Estimating  Function 

To  analyze  the  variation  of  the  estimating  function,  we  introduce 
the  concept  of  jumps  of  the  estimating  function  (see  [22]  and  [24]),  We 
know  that  if 


then.,  because  relation  (1.3a)  is  verified,  there  exists  a  control  u(*) 
such  that 

z(e)  6  W  if  z(0)  =  z  . 

x  -£  o 

o 

However,  relation  (1.3a)  might  be  verified  for  some  smaller  x^  .  If 
X  is  the  smallest  such  x  for  which  it  holds,  we  can  find  a  control 
u°(0  such  that 

z(e)  £  Wt  T^z(e)j  =  xx  -  e  <  xq  -  e  . 
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Let 


We  shall  say  that  with  the  strategy  v(»)  ,  the  estimating  function  has  a 
jump  St  at  zq  .  The  behavior  we  propose  for  the  evader  is  to  make  this 
jump  as  small  as  possible,  and,  if  possible,  zero. 

At  this  point,  a  difficulty  arises.  At  time  Tq  ,  the  evader  chooses 

a  control  v°[t  ,t  +  e]  .  Assume  he  can  choose  it  such  that  the  corres- 
o  o 

ponding  St  be  null.  Let  t^  belong  to  the  interval  (tQ, tQ+e)  .  From 
z(t^)  ,  under  the  previous  controls,  assume  there  exists  a  control 
V'  ^l’  tl+€^  that  avoids  a  jump  in  the  estimating  function  as  well.  We 
are  not  assured  that  v°  and  v'  agree  on  [t^t^+e]  ,  so  that  although 
such  a  control  v*  exists,  the  evader  might  not  be  able  to  use  it  and 
might  let  a  non-zero  jump  occur. 

To  solve,  or  rather  eliminate,  this  problem,  we  use  Fleming's  defini¬ 
tion  of  a  strategy  (introduced  in  [ll]).  We  assume  that  both  players 

choose  their  controls  at  time  t  for  the  whole  interval  (t  ,t  +  e)  and 

o  o  o 

play  them.  Consequently,  the  question  of  updating  the  control  function  at 
an  interior  point  becomes  irrelevant.  This  is  Fleming's  minorant  game 
or  Friedman's  lower  e-strategy  (hence  the  terminology  "lower  rule  e  "). 

A  continuous  strategy  will  be,  by  definition,  the  limit  of  such  a  process. 

It  was  proved  by  Fleming  [12,13]  that  the  value  of  this  discretized 
game  has  a  limit  as  e  goes  to  zero.  Moreover,  this  limit  is  the  same 
as  the  limit  of  the  majorant  game,  defined  in  the  same  way  but  giving  the 
evader  the  information  advantage  (Friedman's  upper  e-strategy).  Fleming's 
proofs  are  made  discretizing  also  the  dynamics,  and  allowing,  then,  for 
mixed  strategies  at  each  move  (and  for  fixed  duration  games).  Friedman 
[14, 15],  generalizing  a  work  of  Vnraiya  and  Lin,  gave  the  proof  using 
continuous  dynamics,  and,  in  [15],  for  a  class  of  games  including  ours, 
with  pure  strategies. 

Therefore,  we  know  our  game  has  a  saddle  point,  and  that  the  limit 
of  the  maximin  capture  time  is  the  saddle  point  value  of  the  game.  From 
now  on,  we  shall  investigate  this  lower  e-strategy.  We  shall  see  under 


23 


what  conditions  the  evader  can  always  prevent  a  jump.  We  shall  take 
t  =  0  since  the  game  is  time-invariant. 


2.4  A  First  Necessary  Condition 

We  are  seeking  an  optimal  e-strategy.  If  a  strategy  v[0,e.]  is 
such  that,  for.  xtO (t  )z  €  W„  , 


o  o 


n4>(t  )z 
o 


.♦■r° 

J  T  - 


t  -e 
o 


<H'r)v(T  -r)dr  €  int 
o 


(v« + 


Prdr  ’ 


then,  by  continuity,  this  is  true  for  some  r  smaller  than  t  ,  and 

o 

there  will  be  a  jump  in  T(z)  .  Thus,  a  necessary  condition  on  v  is 
that 


jM>(t  )z  + 
o  o 


't 


®(r)v(T  -r)dr  6  3<W 

€  °  lV€ 


f° 

J r  -e 
o 


P^dr )  .  (2.4) 


A  necessary  condition  for  this  to  be  always  possible  is 

JT  #*T 

0  4>(r)Qdr  =  W  +  it  °  <D(r)Pdr  . 

T  ~  £  1 

t  -e  o  •'T  -e 

o  o 

Then,  because  itO(T  )z  €  c)WT  ,  WT  is  convex  and  lQ  dr  is  compact, 

J  r 


(2.5) 


there  always  exists  a  control  v  such  that 

•T  . 


Ito (t  )z  + 
o  o 


■r 

J T  -< 


Q(r)v  (t  -r)dr  €  d 
o 


I  W  +  Jt  f  °  $(r)Qdrl 

V.  To  Jt  -€  J 

o 


=  d  (  W  ^  +  it  f  °  O(r)Pdr)  . 

IVs  Jt -e  J 


Proposition:  Condition  (2.5)  implies 
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W  =  C  +  \  P  dr 
T  1  r 


jv  *r 

Jo  Jo 


Q  dr  ,  complete  sweeping, 
r 


Proof:  We  first  prove  that  the  fact  that  it  is  true  for  every  sufficiently 
small  €  ,  for  every  t  ,  implies  that  it  is  true  for  every  €  : 


+[  Q  dr  =  (w  +  (  Q  dr\  +  f 

T  3  t-2  e  r  \  T  3  T-  e  r  /  3  x-2 

rT"e  r 

=  W  +\  Qdr  +  \  Pdr 
t-  e  }  _  r  l  r 

J t-2  e  J t- e 


Q  dr 
r 


■  T-e 

'  t-2  €  J  T-e 

and  this  proves  this  first  claim.  Then,  taking  e  =  t  ,  it  becomes 


rT~fe  r 

=  WT-2e  +  j  ,  Prdr+l  Prdr  ’ 
J  T-2  €  J  T-  < 


W  +  f  Q  dr  =  C  +(  P  dr  , 
T  1  r  1  r 

Jo  Jo 


which  proves  the  proposition. 

In  this  case,  thus,  W^°^  =  .  But  W'  may  be  smaller  or  not 

exist,  since  we  do  not  require  that  P^  3  ,  and  if  it  does,  it  may  not 

have  complete  sweeping. 


2.5  Characterization  of  the  Strategies 

From  the  previous  remarks,  we  can  characterize  the  candidate  optimal 
strategies  v°  and  u°  .  We  need  the  following  simple  result: 

Lemma :  Let  W  be  a  closed,  convex  set  in  a  Euclidean  vector  space.  Let 
A  be  a  closed  set.  Let  £  £  Sw  ,  and  let  n  be  a  normal  to  W 
at  £  (normal  in  the  definition  related  to  convex  sets;  see 
[26]).  Finally,  let  X  €  A  be  such  that 

<n,X)  =  max  <n,n.)  . 

(l  €  A 

Then  X  €  bh  and  j  =  £  +  X  e  d(W+A)  .  Conversely,  if  5  £ 
d (W+A)  ,  there  is  a  normal  n  to  W  at  £  such  that  (n,x)  is 
maximized. 
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Proof:  The  first  part  of  the  claim  is  trivial.  Let  II  be  the  tangent 
hyperplane  normal  to  n  at  £  .  Assume  |  belongs  to  the  in¬ 
terior  of  W  +  A  .  Then,  there  exists  a  neighborhood  of  it  con¬ 
tained  in  W  +  A  ,  which  contains  a  4'  such  that 

(n, !’-{;)  >  (n,4-£)  =  <n,X>  , 

and  since  4'  6  W  +  A  ,  there  exists  a  £'  and  a  X'  such  that, 
V  e  w  ,  X'  €  A  i'  =  V  +  X’  . 

We  have 

<n» X' )  =  <n,6'-C)  =  <n,6'-£)  +  <nf£-C>  . 

Now,  because  W  is  convex  and  n  is  a  normal,  it  is  an  elemen¬ 
tary  property  of  convex  sets  that 

<n,S-S‘>  >  0 

and  consequently 

<n,X’>  >  <n,|'-g>  >{n,x) 

which  is  in  contradiction  with  the  definition  of  X  .  This  proves 
the  direct  part.  Conversely,  let  n  be  a  normal  to  W  +  A  at 
|  ,  Then  (n,4)  is  maximum,  and  thus  both  (n,X)  and  (n,£) 
are.  Hence  n  is  normal  to  W  at  £  ,  and  the  lemma  is 
proved. 

From  this  fact,  we  infer  a  simple  characterization  of  the  strategy 
that  verifies  (2.4)  under  condition  (2,5). 

Let  n  be  a  normal  to  WT  at  t  =  J*<1>(t  )z  .  Any  v  (•)  veri- 
o  o  o  o 

fying 

(n  , rt  \  °  <D(r)v°(t  -r)dr)  =  maximum 

°  -e  J 
o 

verifies  (2.4),  i.e.,  since  the  inner  product  is  linear 

I  °  (n  ,«'I>(r)v0(T  -r))dr  =  maximum  . 

Jt  -€  ° 

o 
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o 

Thus,  v  must  verify  almost  everywhere  in  r  €  [t  -e.T  1 

o  *  o J 

{n  ,  fl<I>(r)v0(T  -r))  =  max  . 

°  °  v  e  Q 

This  implies,  since  the  operator  n®(r)  is  continuous,  that  v°(t)  be¬ 
longs  to  the  boundary  of  Q  for  almost  every  t  .  This  is  a  game  theo¬ 
retic  version  of  Contensou's  optimality  principle.  For  instances  of  its 
use  in  differential  games,  see  [18]  and  [20], 

Another  important  consequence  of  (2.6)  is  that  the  function  v°(0 
is  actually  independent  of  e  .  Consequently,  its  limit  as  e  goes  to 
zero  is  simply  its  limit  as  t  goes  to  zero,  and  is  given  by 

v*  =  Arg  max  (n  ,  )v)  .  (2,6) 

v  £  Q  °  o  ' 

Similarly,  whatever  v(0  is  there  exists  a  normal  n^  to  such 

that  u  verifies 

(n^,  «<l,(r)u0(T^-r)  )  =  max  .  (2.6a) 

Notice  that  if  v  =  v°  ,  then  n  =  n  . 

e  o 

The  function  u°(0  depends  on  e  and  v[0,e]  through  n  .  How- 

6 

ever,  as  e  goes  to  zero,  and  consequently  to  tQ  ,  n^  tends  to  a 

normal  n  to  WT  ,  and  u°(0)  to  an  argument  of  the  corresponding 
o  lo 

maximum: 

u*  =  Arg  max  (n  ,  jM>(t  )u)  . 
u  €  P  °  ° 

In  particular,  if  is  unique,  this  is  independent  of  v  ,  and  uniquely 

defined  if  P  is  strictly  .convex. 

2.6  Technical  Results 

At  this  point,  we  need  some  technical  results.  We  consider  the 
space  K  of  the  convex  compact  subsets  of  L  ,  and,  following  Pontryagin 
[24],  give  it  a  metric  defined  by 

dist  (A,B)  =  max  (dist(a,B)  ,  dist(b,A))  . 
a£A  b£B 
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It  is  easy  to  check  that  this  is  indeed  a  distance.  It  is  stated  in  [24] 
that  with  the  induced  topology,  K  is  complete. 

We  also  introduce  the  notation  C(t)  =  rt$>(T)z  ,  T(z  )  =  t  and 

o  o  o 

for  t  smaller  than  t  let  the  distance  from  ((t)  to  W  be  D(t)  : 

O  'i 

D(t)  =  dist^(T>,wJ  =  min  ||£(t)-t)||  =  ||£ (t)-Y)(t)||  . 

■  T 

t](t)  is  uniquely  defined  in  W^  ,  due  to  the  convexity  of  this  set. 

Lemma  1:  The  vector  T)(t)  is  left  differentiable  at  t  =  Tq  . 

Proof:  Let 

it(w5i[£(VW-!j  £0-C<V 

At)(6t)  =  i  [«V6,)‘0  1o  "  ‘  £„ 

and 

AW(5t)  =  r—  f W  c  -t  1  is  a  convex  set. 

Ot  L  T  “Of  0-1 
o 

Clearly,  A£(8t)  has  a  limit  A£  =  -n$(TQ)Czo  as  St  goes  to 
zero  and 

A£  (5t)  =  A^q  +  0(5t)  . 

We  also  have,  since  t](t)  belongs  to  W^  , 

&I|(5t)  €  AW(6t)  =  (\~V  +  £  C°  5t  Qrdr) 


T 

0 

t  -St 
o 


P  dr  . 
r 


This  expression  for  AW  is  valid  because  W^  verifies  (2.5). 

Let  KTo  be  the  tangent  cone  to  WT^  -  at  the  origin. 

Locally,  the  boundary  of  WT^  -  ^  is  contained  between  KTq 

and  an  arbitrary,  fixed  cone  interior  to  Kto  .  Thus,  locally, 

as  St  goes  to  zero,  (WT  — t  )  can  be  made  arbitrarily  close 

ot  lo  o 


to  KTq  (which  is  invariant  under  scalar  multiplication).  Also, 

the  integrals  can  be  made  arbitrarily  close  to  Q_  and 

To  t0 

Therefore,  AW(5t)  has,  locally,  a  limit  AW  : 


Let  At]q  be  the  closest  point  of  AWq  to  A£q  .  We  shall  prove 
that  it  is  the  limit  of  At)  : 

At](St)  At]^  as  St ->  0  . 

Assume  the  above  statement  is  false.  Then,  there  exists  an  e* 
positive  such  that  for  every  St  ,  there  is  a  St^  smaller  than 
St  ,  for  which 

At)(St1)  =  At)1  I|Ati1-Atjo||  >  e*  . 

Replacing  AW  by  its  tangent  plane  at  An  ,  it  is  easy  to  see 
o  o 

that  this  implies  that  there  exists  a  fixed  0  such  that 

llAt^ll  >  l|AS0-Agi  t  P  .  (*) 

Now,  in  view  of  what  was  said  previously,  we  can  choose  St  such 
that  for  any  St1  <  St  ,  with  A£(5t^)  =  Af^  , 

II^V^II  <  a 

dist(AW,  ,AW  )  <  a  so  that  dist(Aq  ,AW  )  <  a 
1  o  o  T 

then,  since  At^  provides  a  minimum  of  J| A^^— At)||  ,  At]  £  AW(5t)  , 
and  with  the  last  inequality,  and  the  triangular  inequality: 

il^j-^ill  <  Il^-Anjl  +  a 

and  therefore,  with  the  first  inequality 

<  llAC0-An0ll  +  2a  • 

Now  0  is  a  fixed  number  independent  of  St  .  Thus  a  can  be 
chosen  smaller  than  ^0  ,  and  this  gives  a  contradiction  with  the 
inequality  (*).  Therefore  At) (St)  has  a  limit  At]q  ,  and  we  have 
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and  Lemma  1  is  proved. 


Lemma  2:  The  convergence  of  At]  to  At]q  is  Lips  chi  tz  in  T  . 

Proof :  The  proof  is  elementary  and  cumbersome.  It  presents  no  interest 
in  itself  and  will  only  be  sketched. 

Consider  first  the  closest  point  At)*  of  AW^  to  A^  .  It  is 
sufficient  to  prove  the  result  for  the  convergence  of  At]*  to 
At]q  ,  and  then  remark  that  by  the  convexity  of  AW^  ,  At]*  is 
closer  to  At]  than  A£  to  A£q  . 

To  prove  the  result  for  At]*  ,  notice  that  the  convergence  of 

AW  to  AW  is  Lipschitz,  from  the  argument  of- the.  fixed  cones 
t  o 

presented  in  Lemma  1. 

Then  distinguish  between  two  cases: 

e  An  is  not  a  corner  point.  Then  show  that  the  directions  of 
o 

the  normals  to  AW  at  An*  and  to  AW  at  At]  must  agree 
T  o  o 

to  first  order,  which  gives  the  desired  result. 

•  At]  is  a  corner  point.  Then  the  result  comes  from  simple 
geometric  arguments  on  the  farthest  point  where  At]*  can  be, 
knowing  that  the  boundary  of  AW^  lies  within  first  order 
distance  of  the  boundary  of  AWq  . 

In  both  cases,  the  result  is  proved, 

n  (t) 

Corollary  1:  Let  n(t)  =  £(t)  -  t](t)  ,  n(f)  =  ir^yir  ;  then 


n 

o 


dn(r) 

dr 


T=T 

o 


t 


and  if  n  ^  0  ,  n(r)  has  a  limit  n  ,  and  is  Lipshitz,  as 
o  ° 

8t  -»  0  . 


Proof:  The  first  part  of  the  claim  is  trivial. 


Because  of  Lemmas  1  and  2, 


n(T)  =  -n  8t  +  6t  C?(St) 

where  (?(6t)  is  a  vector  verifying  ||c?(6t)||  =  0(6t)  <  p  5t  . 
Thus 

||n (t) ||  =  Hnjlst  +  5t  0(6t) 


and 


n  n  __ 

fi(T)  =  “  urir +  igr 0(5T)  +  ^(8t) 


which  proves  the  claim,  with 


n 

^  o 


Corollary  2:  The  distance  D(t)  is  left  differentiable  at  t  ,  its 

o 

derivative  is 


dP(T) 

dx 


T=T 


=  AD  =  n  . 
o  11  o" 


Proof:  We  have 


D(t)  =  || n. (t) ||  ,  and  AD(8t)  - -  -  D(T) 


6t 


5t 


_  ||  n  (t)  | 

5t 

which,  with  the  calculation  of  Corollary  1,  proves  Corollary  2. 

Remark  1:  It  is  easy  to  see,  by  the  separation  theorem  for  convex  sets, 

that  n(t)  is  a  normal  to  .  Consequently,  if  the  normals  are 

unique,  Lemma  1  can  be  proved  more  directly.  However,  we  want  to  allow 

for  the  case  where  t  is  a  corner  of  WT  ,  and  we  need  Lemma  2. 

o  To 

Remark  2:  A  proof  similar  to  the  one  we  gave  in  Lemma  1,  using  the  same 
tools,  can  easily  be  mode  to  prove  the  convergence  of  n^  to  nQ  in  the 
previous  section. 
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2.7  Sufficiency:  A  Local  Theorem  for  e-Strategies 


Again,  let  z  and  t  be  such  that  T(z  )  =  T  : 

o  o  o  o 

£ (t  )  —  *®(t  )z  €  aw  £(t)  l  W  ,  Vt  <  T  . 

O  OO  T  T  O 

O 

Following  Pshenichnyi  [25],  we  define  a  regular  point  as  a  point  zq 
where 


_d_ 

d-r 


dist 


=  -Ad  ^  o 


T=T 


} 


and  thus,  this  number  is  negative.  We  have  seen  in  the  previous  section 
that  this  derivative  exists. 


Theorem:  If 


z  is  a  regular  point,  under  condition  (2.5)  the  evader 
o 


can  prevent  a  jump  of  the  estimating  function  at  z 

exists  an  €  such  that  for  any  e  smaller  than 
o 

o 

Proof:  We  want  to  prove  that  using  v 
smaller  than 


Precisely,  there 


'M  - 


t  -  e 
o 


defined  in  Section  2.5  there  does 


not  exist  any  t 


i  )z  +  T1 
1  o  1 


To  » 


ST  =  T  -  T,  ,  such  that 

T, 


rt4>(r)v  (T.-e)dr  6  W 
l  T. 


♦f1 

1  G  't  -e 


*<I>(r)Pdr 


rt<J>(r)Qdr  . 


(2.8) 


First,  notice  that  by  definition  of  Tq  ,  5t  goes  to  zero  as  e  does, 
for  any  that  would  verify  (2.8). 

For  every  given  s  €  [0,e]  ,  let  r  =  t.  -  s  and  rQ  =  T0  “  s  • 

Also,  let  v  provide  the  maximum  in  the  following  product  (where 

v  =  JfO(r)v  ) 
r 

max  (v  , n(T. ) )  =  (v*  ,n(T1>) 
v  £  Q  ri  1  rl  1 

where  n(r)  is  defined  as  .in  Section  2.6.  Let  also  n(r^)  =  n^  .  We 

have,  for  every  s  , 


and  we  show  that  the  three  differences  on  the  right  hand  side  are  Lip- 
schitz  in  St  .  It  is  obvious  for  the  third  one,  and  it  is  a  consequence 
of  Lemma  2  in  the  previous  section  for  the  second  one. 

The  first  difference  is  the  variation  of  the  support  function  of 

W  : 

T 

5*(n(T)|wJ  . 

It  is  proved  in  [26]  that  5*  is  convex  in  n  ,  and,  as  such,  Lipschitz 

at  any  point  of  the  relative  interior  of  its  domain.  This,  together  with 

Lemma  2  and  the  obvious  Lipschitz  character  of  its  dependence  on  W  , 

T 

proves  that  this  first  difference  has  the  claimed  property.  Consequently, 
there  exists  an  M  independent  of  6t  such  that 

(v*  ,n  )  -  (v°  ,n  )  <  M5t  Vs  .  (2.9) 

1  1 

Moreover,  remember  that  v°(s)  ,  and  thus  n^  and  v1  ,  are  not  functions 
of  e  .  Thus  M  can  be  chosen  independent  of  e  as  well. 

We  integrate  the  previous  relation  for  s  varying  from  0  to  e  : 


[v1(t1-r)  -  v°(t  -r)]dr 
r  1  v  1 


nJL)  <  M  e  6t  . 


We  subtract  ^(t^)  (defined  as  in  Section  2.6)  from  the  left  hand  side 
of  (2.8),  and  project  on  n^  ,  a  normal  to  WT^  at  ^(t^)  ,  and  compare 
with  the  right  hand  side,  using  the  lemma  of  Section  2.4.  This  gives 


33 


(bCt^)  +  (  1  v° (x^--r)dr  ,  n(t  )) 

J^-e  r 

-  max  (  (  1  v  (t  -r)dr  ,  n(t.)) 

v(-)  JT  -e  r  1  17 

T1  c 

=  D^)  -  {  C  1  [v^('r1-r)  -  v^Ctj-rJJdr  ,  n  ) 

Jv€ 

>  AD  5t  +  0(8t2)  -  MeSt  . 

—  o 

But  e  can  be  chosen  to  make  this  difference  positive  for  every  5t  , 
since  8t  goes  to  zero  with  e  ,  which  proves  that  inclusion  (2.3)  is  not 
verified.  This  proves  the  theorem. 

Remark:  Lemma  2  allows  us  to  use  the  intermediary  of  the  function  5*  , 

and  thus  avoids  an  investigation  into  the  regularity  of  the  function  v  . 

Under  condition  (2.5)  we  have  not  only  proved  that  the  variation  of 
T(z)  is  locally  optimal,  but  also  we  have  the  much  stronger  result  that 
there  exists  an  e-strategy  actually  yielding  this  rate  of  decrease.  This 
may  be  considered  as  very  important  when  it  comes  to  the  implementation 
of  an  optimal  control . 

2.8  Sufficiency:  A  Local  Theorem  for  the  Limit  Process 

In  the  previous  section,  we  were  seeking  an  e-strategy  yielding  the 
time  of  capture  T(z)  .  But  the  existence  of  such  a  strategy  is  not 
necessary  for  T(z)  to  be  optimal.  It  suffices  to  be  able  to  find 
5-efficient  strategies  for  arbitrarily  small  5's  .  Then,  according  to 
our  discussion  of  Section  (2.3),  T(z)  is  optimal  and  corresponds  to  a 
saddle  point. 

Consider  a  trajectory  z(t)  ,  and  let 

z  =  Jt<t>( t  -t)z(t)  z  =  £  €  dW  t(z (0))  =  t 

T  O  T  O  T  \  /  O 

O  O 

and  notice  that,  with  t  -  t  =  t  , 

o 
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z 

=  lim 

i-  U  „  -z 

T 

T  St->0 

A 

St  \  x  -St  t  j 
L  o  o  J 

=  V  -  u 
T  T 

O  O 


Assume  that,  for  every  t  ,  z  €  W 

T  T 


T  -e  <  T  <  T 
O  —  O 


We  have: 

Az  (St)  =  (z-z  )  e  (W-zt  )  =  Aw(8t)  . 

T  OT  T  T  OT  T  T 

o  o 

It  is  easy  to  check  that  dist[AW(&t+e) , AW(5t) ]  goes  to  zero  with  e  . 
The  space  K  beiing  complete,  there  exists  a  limit 

AW  =  lira  f W  ,  -£  )  , 

o  5t_>o  5t  \  tq-5t  o J 

so  that,  in  particular,  our  Corollary  2,  Section  2.6,  holds  without  con¬ 
dition  (2.5).  The  above  inclusion  yields 

v  -  u  6  AW  . 

T  T  O 

o  o 

With  our  definition  of  W  ,  the  pursuer  can  always  achieve  this  goal. 
Thus 


(AW  +P  )  C  Q 
O  T  T 

o  o 


or  0  £  (AW  +P  )  -  Q 
°  T  T 

O  O 


With  these  definitions  and  remarks,  we  prove  the  following  fact: 


Theorem:  At  a  regular  point  zq  ,  if  the  origin  belongs  to  the  boundary 

of  (AW  +P_  )  *  Qt  ,  the  evader  can  insure  that  the  estimating  function 
o  lo  o 

will  have  a  jump  of  the  order  of  e  0(e)  at  most. 

Proof:  Under  the  conditions  of  the  theorem,  there  exists  a  v*  such  that 

vJ  -  uT  cannot  belong  to  the  interior  of  AW 
lo  lo  o 

If  the  pursuer  does  not  choose  u*q  such  that  v*q  €  >  according 

to  our  previous  calculations  z^  cannot  belong  to  W^  for  every  t  in 

a  neighborhood  of  tq  .  Therefore,  let  us  assume  that  u  =  u*  .  Let 

v*  -  u*  =  Ay  €  SAW  . 

T  T  O  O 

O  O 

Ay  is  the  limit  of  a  function  Ay  belonging  to  AW  ,  of  the  form 
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AyT  =  h  W  €  AW(&T) 

yew. 

T  T 

Moreover,  y  can  be  chosen  to  belong  to  the  boundary  of  W  .  Let 
*  T  T 

ZT  be  a  trajectory  generated  by  a  strategy  (u,v)  agreeing  with  u*,v* 

at  t  .  For  instance,  u  and  v  constant.  Then 

h  <*H>>  -  h  ‘W  * °«*> 

and  thus 

z*  =  y  +  St  0(8t)  , 

T  T 

which  proves  that  z*  is  at  a  distance  8t  0(8t)  of  the  boundary  of 

WT  _5t  at  most.  Using  the  assumption  that  z  is  regular,  Ad  is 
o  o  o 

defined  and  non-zero.  The  jump  of  the  estimating  function  is  of  the 
order  of 

ST  disttz  ,dW  )  , 

o 

and  thus,  with  a  given  step  €  ,  this  jump  is  of  the  order  e  0(e)  at 
most.  This  proves  the  theorem. 

Assume,  now,  that  this  holds  in  the  neighborhood  of  a  trajectory, 
except,  possibly,  at  finitely  many  points  on  any  trajectory.  Then,  if  we 
decrease  e  ,  the  number  of  steps  in  a  given  interval  increases  like 
1/e  .  But  if  the  jumps  decrease  as  e  0(e)  ,  the  total  jump  during  that 
time  goes  to  .cero  with  e  .  We  say  that,  locally,  we  have  exhibited  a 
8-efficient  strategy  for  arbitrary  8  . 

What  remains  to  be  done  is  to  see  whether  the  set  family  W^  has 
the  properties  required  by  the  theorem.  We  shall  prove,  in  the  next 
section,  that  Pontryagin's  alternating  integral  verifies  the  following 
relation. 

Proposition: 

K  =  (AW  +P  )  *  Q 

T  O  T  T 
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where  KTo  is  the  tangent  cone  to  WTq  -  £  at  the  origin.  Since 
0  €  ^k-t0  i  our  previous  discussion  holds. 

2.9  Properties  of  the  Alternating  Integral 

To  carry  out  our  program,  we  need  some  preliminary  definitions  and 
results. 

Definition:  A  geometric  difference  C  =  A  *  B  is  said  to  have  complete 
sweeping  (c.s.)  in  the  direction  of  n  when  a  boundary  point  of  C  +  B 
having  this  direction  for  normal  is  also  a  boundary  point  of  A  . 

Notice  that  then  all  such  points  will  have  that  property.  Notice 
also  that  at  every  boundary  point  of  C  there  is  at  least  one  normal 
having  c.s.  in  its  direction. 

Lemma:  For  every  set  A  ,  B  and  C  for  which  this  combination  exists, 
we  have 

£a+B)-C)+B)-C  =  (A+2B)  *  2C 

where  the  notation  of  the  left  hand  side  has  an  obvious  meaning. 

Proof:  From  the  results  of  Section  1.6,  we  would  have  the  left  hand  side 
included  in,  or  equal  to,  the  right  hand  side.  To  prove  the 
equality,  we  prove  that  any  boundary  point  of  the  left  hand  side 
is  a  boundary  point  of  the  right  hand  side. 

To  do  so,  we  prove  that  all  three  geometric  differences  have 
complete  sweeping  in  any  direction  in  which  the  last  one  of  the 
left  hand  side  has.  Once  this  is  proved,  the  result  follows 
rapidly: 

Let 

D  =  (A+B)  *  C  D  =  (D  +B)  -  C  D  =  (A+2B)  -  2C  . 

1  «  A 

Consider  a  boundary  point  d^  of  ,  and  a  normal  n  to  D 2 
at  d0  such  that  (D  +B)  -  C  has  c.s.  in  its  direction.  Con- 
sider  then 
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Arg 

max 

(n,b) 

b  e  B 

Arg 

max 

(n, c)  . 

c  €  C 

Because  of  the  hypothesis  of  c.s.  in  that  direction,  we  have 
d  +  c  £  d(D  +B) 

m  J. 

and  as  n  is  also  normal  to  +  B  at  d^  +  c  , 

d^  =  d2  +  c  -  b  £  SD  ,  n  normal  to  at  d^  . 

By  the  same  reasoning,  we  deduce  that 
d1  +  c  -  b  €  dA 

with  d^  =  d2  +  2c  -  2b  ,  and  again  using  the  fact  that  n  is  a 
common  normal,  that  this  implies 

d  €  d[(A+2B)  *  2C]  . 

Therefore,  the  only  thing  we  have  left  to  prove  is  the  following 
proposition: 

Proposition:  Let  n  be  a  direction  in  which  (D^+B)  -  C  has  c.s.,  then 
(A+B)  *  C  and  (A+2B)  *  2C  have  c.s.  in  that  direction. 

Proof :  If  we  replace  the  first  set  of  a  geometric  difference  by  a  set 

which  has  at  every  corresponding  point  of  its  boundary  (common  normal)  a 
bigger  radius  of  curvature  of  "less  acute"  corner  points  (larger  cone  of 
normals),  no  direction  can  lose  its  c.s.  property. 

[ (A+2B)  *  C]  has  this  relationship  with  [^(A+B)  -  cj  +  B]  ,  thus, 
in  a  direction  n  where  (D^+B)  -  C  has  c.s.,  [ (A+2B)  *  C]  -  C  has, 
too.  Now,  notice  that  (see  Section  1.6)  [(A+2B)  -  C]  -  C  =  (A+2B)  *  2C  . 

Take  a  boundary  point  d  of  this  set,  where  n  is  a  normal.  Let 
c  €  C  maximize  the  inner  product  (n,c)  .  Then 

d  +  c  £  d[(A+2B)  2  c]  , 

and  comparing  the  normals,  it  follows  that 


d  +  2c  €  d(A+2B)  . 


Thus,  (A+2B)  *  2C  has  c.s.  in  the  direction  of  n  .  Then,  because  of 
our  introductory  remark,  (2A+2B)  *  2C  has,  too.  And  by  mere  similitude, 
(A+B)  1  C  as  well. 

Thus,  the  proposition  is  proved,  and,  consequently,  the  lemma. 

Now,  we  can  prove  the  last  proposition  of  the  previous  section. 

Notice  that,  by  induction,  this  property  is  true  for  any  alternating  sum 
of  two  sets  of  the  form 

(A+B)  *  C)  +  B)  *  C _ )  +  B)  *  C  =  (A+nB)  -  nC  n  =  2P  . 

Consider  the  set  WTq  defined  by  the  alternating  integral 


[W  -£  ]  C  (jL  (W  -5  )  +  i  (  °  P  dr\ 
8t  to  o  \St  to-6t  ^o  St)t_&t  r  j 

o 


*  i_  r° 

"  5t  . 


T  -5t 
o 


Q  dr  . 
r 


The  left  hand  side  can  be  made  arbitrarily  close  to 


rl'(w  R"0+r’(°  ~n  Pdr)ir'(0  n  Qdr) _ 

L5t  V  V5t  ol  5T  -6t  r  /  5T  Jt  -8t  r  / 


J_  (T°  P  dr)  -*  1  r° 

r  /  6t3t.i 


n  =  2l 


1  6t 
o  n 


which,  by  continuity  of  the  geometric  difference  for  convex  sets  (see 
[24])  can  be  made  arbitrarily  close  to 
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This  set,  in  turn,  can  be  made  arbitrarily  close  to  the  right  hand  side 
of  the  above  inclusion.  Thus,  the  distance  between  the  two  sides  of  this 
inclusion  goes  to  zero  with  St  .  and  we  have  the  property  claimed: 


=  (AW  +P  )  S  Q 
O  T  T 


> 


the  existence  of  the  other  limits  proving  that  AWq  £  E  .  Notice  that 
AWq  is  not  necessarily  a  cone,  but  has  KTq  as  its  recession  cone.  The 
above  geometric  difference  does  not  necessarily  have  complete  sweeping. 

It  has  under  condition  (2.5)  as  we  saw  in  Section  2.6. 


2.10  Sufficiency:  A  Global  Condition 

In  this  section,  we  shall  assume  without  proof  that  the  strategies 
u*  and  v*  only  have  isolated  simple  jumps,  so  that  there  always  exists 
a  left  continuous  definition  of  them  at  any  point. 

Should  this  be  not  true  at  some  point,  only  the  strong  version  of 
the  condition  derived  would  hold,  and  it  would  no  longer  imply  the  weak 
one.  Notice  that  if  such  a  behavior  happened  at  more  than  isolated 
points  of  a  trajectory,  we  could  always  replace  the  "chattering"  control 
by  an  equivalent  non-chattering  one,  due  to  the  convexity  of  the  control 
sets . 

i)  The  Problem.  We  have  seen  that  under  condition  (2.5),  if 

T(z  )  =  t  ,  there  exists  an  e°  such  that  T  z(e°)  =  t  -  e°  .  Let 
o  o  o 

z(e°)  =  z'  ,  T(z')  =  t  -  e  ,  and  there  exists  an  e'  having  the  same 

property,  etc.  However,  what  may  happen  is  that 

e1  =  e  <  r  r  -  e  =  t°  >  0  . 

o  o 

Then,  the  point  z°  =  z(e)  is  such  that 
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-  T*  =  St  >  0 


(2.10) 


Jt<t(T°)z°  €  dw 

T° 

«4(t'  )z°  €  , 


or  a  non-regular  point.  Otherwise  our  local  proofs  would  hold  in  a  neigh- 
o 

borhood  of  z  ,  in  contradiction  with  the  hypothesis.  If  we  rule  out, 
by  assumption,  non-regular  points  of  second  kind  (see  next  section),  then, 
as  we  shall  see  in  the  next  proposition,  relations  (2.10)  hold  at  z°  . 

Similarly  for  the  case  of  the  continuous  process,  we  have  seen  that 

a  jump  would  be  of  the  order  of  e  0(e)  because  the  distance  of  z  to 

T 

the  boundary  of  is  of  that  order.  But  the  proof  fails  if  z(t) 

comes  arbitrarily  close  to  a  point  verifying  (2.10). 

Therefore,  we  must  impose  some  conditions  on  points  of  this  type. 

And  since  z°  can  be  approached  arbitrarily  closely  on  the  trajectory 
without  jumps,  the  only  points  to  consider  are  those  of  the  following 
set  F  : 


F  =  d([#(-T°)Jt*1dW^)]  n[4>(-T')Jf1dWT,]}  t°  -  r*  =  6t  >  0 

being  understood  that  this  boundary  is  to  be  considered  only  where  it 
separates  the  intersection  from  a  region  where  T(z)  is  in  the  neighbor¬ 
hood  of  t°  . 

Proposition:  F  is  the  union  of  parts  of  the  boundary  of  C  ,  and  of  loci 
of  non-regular  points  of  first  kind. 

Proof :  We  assume  that  6  dW^,  ,  but  an  arbitrarily  close  point 

z\q  does  not  belong  to  any  W  ,  ,  with  t'  in  the  neighborhood  of  r’  . 
This  can  happen  in  two  ways: 

•  Either:  for  €  sufficiently  small,  locally  we  have 

aw  C  w  0  VT  €  (T°-e,T°+e)  . 

T 

Then  it  is  easy  to  see  that  z°  is  non-regular  at  t'}  moreover, 

z° ,  belongs  to  the  envelope  of  the  W  's  ,  which  will  be  seen  to 
T  T 

be  the  characteristic  property  of  non-regular  points  of  first  kind. 

•  Or  else:  W  is  not  defined  in  an  open  neighborhood  of  t'  .  But 

T 
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if  it  is  defined  and  has  an  interior  for  t'  ,  it  is  defined,  by 
continuity  for  some  larger  t  .  The  only  possibility,  thus,  is 
that  t'  =  0  so  that  W  is  not  defined  for  t  <  T1  .  Then 

V-c. 

Therefore,  the  problem  is  reduced  to  checking  whether  the  trajectories  we 
have  defined  penetrate  such  surfaces.  In  the  absence  of  non-regular 
points  of  second  kind,  we  have  the  following  result: 

Theorem:  The  necessary  and  sufficient  condition  for  the  estimating  func¬ 
tion  to  be  optimal  is  that  the  corresponding  trajectories  do  not  cross 
the  manifold  F  . 


ii)  Sufficient  Conditions.  Sufficient  conditions  can  be  derived  on 
the  structure  of  the  game,  not  requiring  the  actual  computation  of  the 
trajectories.  Notice  that,  in  principle,  once  the  sets  are  known, 

the  manifold  F  is  known. 

A  first,  obvious,  sufficient  condition  is  F  =  0  .  However,  this 
rarely  happens,  although  one  could  construct  examples  that  satisfy  this 
condition. 


We  can  deduce  different  conditions  from  another  idea:  it  suffices 
to  insure  that  a  trajectory  arriving  at  z°  would  lie  in  vTi_£  a  time 
e  earlier.  Then,  there  is  no  jump  in  T(z)  at  z°  ;  thus  the  problem 
mentioned  does  not  occur.  This  is  what  we  shall  call  "condition  B." 

It  is  insured  by  the  following  strong  version: 

Let  z°  €  F  ;  there  exists  a  neighborhood  of  z°  for  which,  if 

z  „  6  dW  .  ,  there  is  a  normal  n  to  W  at  z  such  that 

T°+e  -r°+e  e  t°+e  x°+e 

the  corresponding  u°  and  v  verify,  for  every  normal  n'  to  W^t 

.  o 
at  z  , 

T 


(n',v°(s)  -  u°(s))  >  (n',v^(s)  -  u'^s)) 

Vt  =  t'  +  e  -  s  0  <  s  <  e 

u°,v°  ,  with 


(2.11) 


where  u'(*)  and  v’(0  are  defined  similarly  to 
t'  .  This  condition,  directly  derived  from 


n'  and 


actually  means  that,  judged  according  to  the  sets  V  ,  ,  the  strategies 
(u°,v°)  are  not  worse,  for  the  evader,  than  the  optimal  pair  (u',v')  . 

This  condition  is  still  complicated,  but  two  interesting  forms  can 
be  derived  from  it,  easier  to  check. 

The  first  one  is  condition  A  of  Gusyatnikov  and  Nikolsky  (see  Section 

o  o 

2.2).  In  that  case,  a  u  corresponds  to  v  such  that 


u°  -  v°  £  P  S  Q  Vt 

T  T  T  T 


insuring  that  condition  B  is  satisfied. 

The  superiority  of  this  condition  is  that  it  comes  the  closest  to 

dealing  with  the  raw  data  of  the  problem.  This  point  is  investigated  in 

more  detail  in  [l6l.  Its  main  restriction  is  that  it  requires  P  *>  Q 
J  T  T 

for  every  t  ,  which  gives  the  pursuer  an  excessive  superiority  over  the 
evader. 


Another  form  is  the  weak  version  of  (2.11),  valid  with  our  assump¬ 
tion  on  the  regularity  of  the  optimal  strategies.  Then  (2.12)  is  insured 
by 


>  <■>' 


v 

T 


i 


(2.11) 


which  can  be  derived  as  a  limit  of  (2.11)  or  by  an  argument  similar  to 
that  of  Section  2.8.  Notice  that  if  we  allow  "safe  contact,"  then  the 
strict  inequality  in  (2.11)  can  be  replaced  by  "greater  than  or  equal  to." 

Finally,  this  is  verified  if  v*  can  be  determined  as  a  function  of 
z  only,  independent  of  t  .  Then,  v*  =  v1  ,  and  as  u'  provides  a 
minimum  in  the  expressions  of  (2.12)  (or  (2.11)),  condition  B  is  satis¬ 
fied.  This  form  is  also  a  structural  condition,  not  requiring  that  F 
be  explicitly  found. 


2.11  Non-Regular  Points 

i)  Charqcteriza tion.  Since  we  have  been  obliged  to  assume  that  all 
the  points  of  our  trajectories  were  regular,  it  is  interesting  to  see  in 
more  detail  what  happens  at  a  non-regular  point.  (We  prefer  to  keep  the 
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terminology  singular  for  another  type  of  point  we  shall  introduce  in 
Chapter  5.) 

A  non-regular  point,  we  recall,  is  a  point  where 

-f-  distf^(r),W  )  =0  £(t  )  =  Jt$(T  )z  €  BW 

dr  \  T/  o  o  o  x 

T=T  O 

O 

£(t)  t  ,  VT  <  To 

We  can  easily  verify  the  following  fact: 

Proposition:  At  a  non-regular  point,  the  gradient  of  the  estimating 
function  is  infinite. 

Proof:  Consider  an  inverse  image  z'(t)  of  t)(t)  by 

z'(t)  €  <I>(-T)Jt~1T)(T)  T^z’(t))  =  T  . 

It  is  possible  to  choose  it  in  such  a  way  that  the  limit  of  the  line 
(zq,z')  does  not  reduce  to  when  acted  upon  by  the  operator  . 

Thus,  the  length  of  h(-r)  =  z'  -  zq  verifies 

II h <t) !|  =  M(t)D(t)  with  M(t)  <  M  as  t  ->  t  . 

Now,  as  h  goes  to  zero,  we  have 

T(z)  -  T(z')  =  5‘f  =  <VT(z),h>  +  0(6t2) 
where  Vr(z)  is  the  gradient  of  T(z)  .  Now,  let  fi  =  , 


<7r(z),l») 


5t+0(5t2)  1 

M(t)  D(t)  — 


+  0(5t) 


Thus,  if  AD  is  zero,  the  inner  product  is  infinite,  which  proves  the 
o 

proposition. 

ii)  Classification.  To  go  further  in  the  analysis,  it  is  convenient 
to  distinguish  between  two  kinds  of  non-regular  points: 

•  First  kind:  £(t)  does  not  penetrate  W^  ,  and  more  precisely, 
there  exists  a  positive  e  such  that 

$(t>  <L  w  Vt  e  (T  -6,T  +£)  T  ^  T  . 
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•  Second  kind:  the  above  property  is  not  verified.  £(t)  may  either 

belong  to  the  boundary  of  W  for  a  finite  interval  in  r  ,  or 

t 

penetrate  into  the  interior  of  W 

x 

The  interest  of  this  classification  appears  in  the  following  fact: 

Proposition:  A  non-regular  point  of  first  kind  lies  on  the  envelope  of 

the  sets  V  . 

T 

Proof:  This  immediately  follows  from  the  definition,  transformed  in 

terms  of  z  and  V 
o  T 

This  envelope  is  clearly  a  discontinuity  in  the  function  T(z)  . 

This  is  consistent  with  our  remark  that  the  gradient  of  T(z)  is  in¬ 
finite.  Actually,  the  envelope  is  a  barrier  according  to  Isaacs.  What 
we  have  here  is  a  mere  statement  of  Isaacs'  envelope  principle  (see  [18]). 

iii)  Properties  of  Barriers.  Such  an  envelope  is  a  closed  manifold 
(see  also  [25]).  Thus,  if  a  trajectory  reaches  it  from  regular  points, 
the  set  of  regular  points  on  this  trajectory  is  open. 

Moreover,  if  this  trajectory  comes  from  "outside"  the  barrier, 
namely,  from  the  exterior  of  the  union  of  sets,  the  envelope  of  which  is 
the  barrier,  then  it  has  a  finite  jump  in  the  estimating  function. 

But  because  of  the  previous  remark,  the  proof  of  Section  2.8  holds 
along  the  trajectory,  yielding  the  following  result: 

Theorem:  Under  condition  B,  trajectories  generated  by  the  limit  process 
never  cross  a  locus  of  non-regular  points  of  first  kind  that  would  induce 
a  jump  in  the  estimating  function. 

Finally,  we  have  the  following  result,  not  really  needed  in  a  theory 
of  the  optimality  of  the  process,  but  interesting  because  it  corresponds 
to  the  cases  which  are  usually  met: 

Theorem:  If  along  a  barrier  as  defined  in  this  section  the  normal  to 

W  is  unique,  then  a  trajectory  generated  by  the  limit  process  having  a 
T 

point  in  the  barrier  lies  completely  in  it. 

Proof:  At  a  point  where  the  normal  to  WTo  is  unique,  the  cone  KTo 
is  a  half  space,  and  its  boundary  the  hyperplane  tangent  to  W1q  .  Then 
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Mw  is  a  hyperplane  parallel  to  cSKT  . 
o  lo 

The  fact  that  the  point  is  non-regular  means  that 

AD  --An  -  At  =  0  . 
o  o  3o  7 

and  since  At]  lies,  by  definition,  on  SAW  ,  this  is  true  also  of 
o  o 

A^q  .  Now,  consider 

tcO(t  )z(0)  =  )  (cz  -u*+v*)  =  v*  -  u*  -  At 

O  O  O  x  T  O 

O  O 

If  v*  and  u*  are  chosen  according  to  (2.7)  and  (2.7a),  vj  -  uj 

lo  lo 

belongs  to  SAW  .  As  A£  does,  too,  we  see  that  jt<Kt  )z(0)  is  either 
o  o  o 

zero  or  parallel  to  Skt  ,  and  xhus  to  Sw_  . 

o  lo 

In  both  cases,  this  implies  that  z  is  parallel  to  SV-.  .  As  we 

o  To 

know  that  zq  lies  on  the  envelope  of  the  family  ,  we  see  that  under 
the  continuous  law,  z  remains  on  this  envelope. 

This  proves  the  theorem. 

iv)  Non-Regular  Points  of  Second  Kind.  Non-regular  points  of 
second  kind  appear  as  points  where  the  estimating  function  has  an  in¬ 
finite  gradient  without  being  discontinuous. 

We  propose  the  following  example,  which  shows  that  such  points  can 
exist,  and  gives  seme  indication  of  what  they  actually  represent. 

Consider  a  two-dimensional  game  where  the  geometrical  space  is  the 
whole  state  space;  therefore  n  =  I  the  identity.  Let  the  dynamics  be 
defined  by 


/-a 

-0)\ 

c  =( 

a,<u 

positive  real  numbers 

'  CD 

-a/ 

and  P  =  Q  ,  so  that 

P  -  Q 

T  T 

=  (o) 

.  Finally,  let  the  capture  set  C 

be  the  disk  centered  at  the  point 


and  of  radius  S,  =  so  that  its  boundary  goes  through  the  point 

(1,0)  . 


46 


Since  1  =  {0}  ,  is  constant  and  equal  to  C  :  with  the 

proposed  strategies,  the  two  players'  actions  cancel  each  other  and  the 
state  follows  the  free  dynamics  of  the  system.  The  transition  matrix 
of  C  is 

C^rX'r  , 

Q  ’  C03  ODT  —  C  3%U  W  ■. 

' 

-a t  ,  -ax 

e  sin  cdt  e  cos  cot 

so  that  for  a  fixed  z  ,  £(t)  =  ^(xiz  describes  a  logarithmic  spiral 

as  t  varies. 


The  capture  circle  has  been  chosen  such  that  it  is  the  osculating 
circle  to  the  spiral  through  that  point  at  <2=1,  y=0.  As  a  conse¬ 

quence,  this  whole  spiral,  outside  of  C  ,  is  a  locus  of  non-regular 
points  of  second  kind. 


In  fact,  for  a  point 


we  have 


z  =1 


CCS 

e  cos  cos 


CCS 

i-e  sin  cosy 


and  thus 


£(t)  = 


-a(T-s)  , 

e  cos  co(t-s) 


-a(x-s)  ,  . 

>e  .  sin  co(t-s)> 


d(t) 


a/2 


-2a<T-s)  2a  -a(t-s)  . 

e  +  —  e  sin  co(t-s)  + 

co 


v/C^-KO2 


CO 


47 


It  is  a  simple  matter  to  check  that 


1) 

for 

x  <  s 

d(t)  >  0  , 

and  D(s)  =  0 

2) 

for 

II 

c n 

dD(x) 

dx 

3) 

for 

T  =  S 

d2D(x)  Q 
2 

and  d  D(3T)  =  -CtoN/o?^2  <  0  , 

dx  dr 


establishing  that 

1)  T(z)  ^  s 

2)  z  is  a  non-regular  point 

3)  it  is  a  non-regularity  of  second  kind 
which  is  what  we  wanted  to  show. 

Notice  that  T(z)  =  s  proves  that  the  gradient  of  the  estimating 
function  has  a  finite  component  tangent  to  the  spiral.  Since  this  gradi¬ 
ent  is  actually  infinite,  it  is  normal  to  the  spiral. 

This  finishes  our  discussion  of  non-regular  points  of  second  kind. 

2.12  Conclusion 

o  o 

We  have  characterized  directly  the  controls  u  and  v  ,  and  found 
a  direct  construction  of  their  limits  u*'  and  v*  . 

An  interesting  feature  is  that  while  u°  is,  under  some  conditions, 
optimal  against  every  v  ,  it  was  found  that  its  limit  u*  often  does  not 
depend  on  v  .  This  is  true  if  the  sets  do  not  present  corner 

points,  except,  possibly,  for  countably  many  values  of  x  . 

Notice  also  that 

(n,  rt'Kx)  (v-u) )  =  j*n,  v-u) 

where  ^^(x)^*  is  the  adjoint  operator  to  rt<J>(x)  •  Let,  then,  \  = 
(«<I>(x)j  n  ,  and  it.  is  seen  that  the  controls  u*  and  v*  must  be  such 
that 


(\,2(u*,v*))  =  min  max  (x.,z(u,v)) 
u  v 

which  bears  a  close  resemblance  to  the  Pontryagin  Maximum  Principle. 
Because  of  the  possibility  of  the  occurrence  of  corner  points  in  the 
sets  ,  the  variation  of  \  may  be  difficult  to  describe.  X  may  even 
be  non-unique.  And  it  is  noteworthy  that  the  geometric  subtraction  can 
introduce  corners  without  any  of  the  constituting  sets  having  one  (and 
still  without  violating  condition  (2.5)). 

With  regard  to  the  question  of  the  optimality  of  the  process  de¬ 
scribed,  we  have  found  that  under  condition  (2.5)  the  time  T(z)  can  be 
optimal  for  an  e-strategy,  and  a  fortiori,  of  course,  for  the  limit  pro¬ 
cess.  But  this  condition  is  not  needed  for  the  limit  process,  and  we 
have  found  that  the  alternating  integral  is,  as  far  as  local  behavior  is 
concerned,  the  optimal  capture  set. 

However,  the  corresponding  trajectories  can  still  fail  to  be  op¬ 
timal  by  crossing  a  barrier  or  penetrating  the  "non-usable  part"  of  the 
capture  set.  Actually,  in  all  instances  known,  it  is  the  second  phe¬ 
nomenon  that  occurs.  As  will  be  seen  in  the  second  part,  this  leads  to 
state  constrained  optimal  trajectories  along  a  "safe  contact."  Condition 
B  is  sufficient  to  prevent  this  from  happening. 

In  addition,  all  sufficiency  conditions  must  exclude  non-regular 
points  of  second  kind.  Apart  from  that,  all  conditions  are  both  neces¬ 
sary  and  sufficient. 
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3.  MULTISTAGE  GAMES 


In  this  chapter,  we  consider  multistage  games,  that  is,  games  in 
which  the  system  to  be  controlled  is  in  discrete  time,  governed  by  a 
difference  equation.  We  shall  briefly  discuss  the  discrete  equivalent  of 
system  (1.1),  and  then  turn  to  the  system  theoretic  formulation,  with 
unbounded  controls,  for  which  the  present  technique  turns  out  to  be  par¬ 
ticularly  well  adapted. 

3.1  The  Discrete  Game 

In  a  very  classical  way,  the  system  (1.1)  can  be  transformed  into  a 
discrete  one,  letting 

z(ne)  =  z(n)  e^  =  $(e)  =  $ 

yields 

z(n+l)  =  <&z(n)  -  u(n)  +  v(n)  (3.1) 

where 

u(n)  €  P  v(n)  £  Q  . 

P  and  Q  are  compact  convex  sets  derived  from  the  original  one  In  a 
trivial  way.  Define 

P  =•  rt<J>np  Q  =  n<D?Q  . 

n  n 

And  as  in  the  first  chapter,  consider  the  sets 

W  = 
n 

w(oo) 

n 

and  the  sets  V 

n 

V  = 
n 

v(“) 
n 


((•••  %)  +  Pl)  -*  Q1  "•  )  +  Vl)  2  Vl 

=(c+8p*)-*£q‘ 

( oo) 

and  V  defined  by 
n 

(z|rt<fcnz  €  Wn) 

=  (z|it$nz  6  }  ’ 

V  and  V(eo) 
n  n 


And  we  claim  that 


are  the  sets  of  capturable  points, 


respectively,  when  v(n)  is  known  of  the  pursuer  as  step  n  ,  and  when 
the  whole  future  history  v(*)  is  known. 

The  proof  for  V  is  rigorously  the  same  as  in  Chapter  1;  there  is 
n  (co) 

no  need  to  repeat  it.  We  prove  the  claim  for  since  the  situation 

is  slightly  different:  we  add  steps  together  instead  of  letting  the 
step  grow  up  to  the  capture  time. 

We  have 

rtz(n)  =  rt<2>nz(0)  ~  u^ 

where 

A  ,k 
u  =  Jt4>  u 
k 

For  simplicity  of  notation,  let 

=  §uk<n-1-k)  ‘Pn  €  £Pk 

♦n  ■  Up  V"-1-*0  e  |X  • 

For  capture  to  be  possible  in  n  steps,  it  is  necessary  and  sufficient 

that  there  exist  a  q>  such  that 

Tn 

Jt<Dnz(0)  -  cp  +  ijr  €  C  , 
n  n 


(n-l-k)  +  vk(n-l-k) 


or  equivalently,  that 


Jt<Knz(0)  +  *  S  C  + 
n 


And,  for  this  to  be  possible  for  every  ,  it  is  necessary  and  suffi¬ 
cient  that 


JtGnz(0) 
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or  equivalently  that 


w”2(°)  s  (e  +  X1  P»)  '*  ?  Q|t  ' 

which  proves  the  claim. 

3.2  Capture  with  No  Information  on  v 

In  the  continuous  case  we  had  the  possibility  of  letting  e  go  to 
zero.  Here,  if  we  want  to  have  the  information  advantage  of  the  pursuer 
vanish,  the  only  thing  we  can  do  is  assume  that  he  has  no  information  on 

v  .  Then,  as  we  want  capture  to  be  possible  whatever  v  is,  it  must  be 

possible  if  the  evader  plays  "as  if  he  knew"  the  pursuer's  control.  Thus, 

we  are  actually  led  to  the  study  of  the  majorant  game,  which  was  not 

needed  in  the  continuous  case. 

By  analogy  with  the  previous  constructions,  we  are  looking  for  a 

set  ft  such  that 
n 

Jt<i>nz(0)  €  ft  (3-2) 

n 

insures  that  it  is  always  possible  for  the  pursuer  to  obtain 

rt<Dn“‘1z(l)  e  ft  (3.3) 

and,  in  addition, 

ft  =  C  . 

Inclusion  (3.3)  reads 

nfcnz(0)  -  u  (0)  +  v  .  (0)  €  ft  V  v  €  Qn  .  . 

n-1  n-1  n-1  n-1  n-l 


Therefore 


Jt0nz(0)  -  u  _  (0)  +  Q  C  ft  ; 

n— l  n-i  n-i 


equivalently 


n«l> 


'*»«)  -  vi(0)  G  Vi s  Vi  • 
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The  existence  of  such  a  u  is  equivalent  to 


*$nz(0)  £  (ft  i  Q  )  +  P 

n-1  n-1  n-1 

and  as  this  must  be  a  consequence  of  (3.2), 


ft  £  (ft  ,  4  Q  ,)  +  P  ,  . 
n  n-1  n-1  n-1 


(3.4) 


(3.4a) 


More  precisely,  as  (3.4)  is  a  necessary  and  sufficient  condition  to  in¬ 
sure  that  (3.3)  is  possible,  we  replace  (3.4a)  by  the  equality,  and 
using  it  recursively  together  with  ftQ  =  C  ,  define  ft^  : 


ft 

n 


((-•-((( e 25  0 *  0  -*  QJ +  pi)--) -*  Vx) +  Vi  • 


Remarks 

As  could  be  expected,  ft  does  not  exist  unless,  in  particular, 

CD  Q  .  The  relative  size  of  the  three  sets  W  is  easy  to  check: 
o  n 

ft  Cw  cw(M) 

n  n  n 

by  straightforward  application  of  the  propositions  of  Section  1.6.  They 
may  also  be  used  to  establish  that  if  P  ®  Qj[  ,  for  every  non-negative 
i  ,  with  P  a  C  ,  and  if  P^  -  Qi  has  complete  sweeping,  then 

ft  =  w  =  w(oo)  . 

n  n  n 


3.3  Concluding  Remarks 


We  have  not  said  anything  about  optimality  so  far. 

In  the  case  of  V(cc)  ,  later  referred  to  as  the  strong  controlla- 
n  n 

bility  case,  we  have  seen  that  z(0)  £  to  insure  z(l)  £  'rn_^  • 

This  corresponds  to  the  local  theorems  of  Chapter  Two.  But  we  have  the 
same  "global"  problem.  We  are  not  sure  that  the  evader  can  simultaneously 
prevent  the  state  from  penetrating  every  of  smaller  index. 

One  should  therefore  either  find  an  equivalent  of  condition  B,  but 
this  could  be  more  difficult  than  in  the  continuous  case,  or  redefine  the 
family  V  in  such  a  way  that  includes  all  points  z  such  that 
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the  evader  cannot  prevent  the  state  from  drifting  in  one  step  into 
simultaneously  with  the  n-1  previous  sets  . 

We  did  not  investigate  this  problem.  We  notice  only  that  if  V  D 
V  for  every  n  ,  then  the  problem  does  not  arise,  and  the  previous 
construction  yields  the  optimal  capture  time. 


3.4  System  Theoretic  Formulation 

We  turn  now  to  the  discrete  system  with  unbounded  controls.  We  must 
obviously  reintroduce  the  matrices  G  and  J  through  which  they  act. 

We  also  change  our  notations  to  more  traditional  ones. 

We  deal  with  the  system 

x(k+l)  =  F  x(k)  -  G  u (k)  +  J  v(k)  (3.5) 

where 

x  €  X  an  n-dimensional  vector  space 

F  is  an  nXn  constant  matrix 

u  €  U  an  m-dimensional  vector  space 

v  €  V  an  m' -dimensional  vector  space 

G  and  J  are  nXm  and  nXm'  constant  matrices. 

A  subspace  M  of  X  is  given,  and  capture  is  defined  as  x  €  M  .  We 
choose  a  complement  L  of  M  :  L  ©  M  =  X  and  we  define  k  as  the  pi’c- 
jection  onto  L  parallel  to  M  : 

jfx  €  L  x  -  fix  6  M 


and  capture  is  equivalent  to  J*x  =  0  . 

We  introduce,  in  addition,  the  notations 


P  =  Jt  range  (f'xG) 

K 


Q  =  a  range  }  . 

K 


P  and  Q  are  vector  subspaces  of  L  . 
k  k 

We  can  still  define  a  geometric  subtraction:  given  two  subspaces 
A  and  B  , 

A  -  B  =  D  =  {x|x  +  B  C  A)  . 
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But  this  operation  is  now  particularly  simple.  Two  cases  arise: 


A  2  B 
A  £  B 


A  -  B  =  A  with  complete  sweeping 
A  -  E  =  0  or,  equivalently,  does  not  exist. 


And  finally,  from  the  Cayley-Hamilton  Theorem, 


P ■  • 


ange  {^G,  «FG,  ...,  JQrG)  Q 


and  similarly  for  the  Q^'s  ,  so  that  we  can  stop  all  our  constructions  at 
n  steps. 


3.5  Strong  Controllability,  Capturability  and  Ideal  Capturability 

We  apply  the  same  technique  as  previously,  with  vector  subspaces. 
We  again  have  the  three  main  information  structures: 

a)  Strong  Controllability.  The  control  v(k)  is  known  for  the 
whole  future.  If  the  state  can  be  brought  to  the  origin,  we 
shall  say,  following  Kalman  [19],  that  it  is  strongly  control¬ 
lable  modulo  M  . 

We  have 


Therefore 


if  !|!pi-2!|!Qi 
“k"’  - 0  u  ^  pi 2  §  • 


The  condition  for  the  existence  of  W 


can  be  written  as  an 


explicit  condition  on  the  coefficients  of  the  matrices  involved: 
rank  [  JtG  ;«FG  ■ . . .  «Fk  ^G]  =  rank  [  rtG  j  itJ  j  JtFG » JtFJ  [••• 

itFk-1G  !nFk-1J]  . 
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f  (  °°) 

Notice  that  W  may  exist  (be  non-empty)  while  a  set  of  lower 

K  (cc) 

order  would  not.  The  states  of  the  corresponding  V  are 

K 

still  strongly  controllable,  since,  as  we  saw  in  Section  3.1,  the 
argument  for  that  case  does  not  proceed  by  induction. 

We  find  that  for  every  k  ,  either  all  states  controllable  with 
u  alone  in  exactly  k  steps  are  strongly  controllable  in 
exactly  k  steps,  or  none  is.  In  the  latter  case,  however, 
some  states  may  still  be  strongly  controllable  in  less  than  k 
steps.  But  the  pursuer  will  not,  then,  be  able  to  keep  the 
state  in  M  until  time  k  ,  or  to  have  it  reach  M  at  time  k 
only. 


0)  Capturability .  The  value  of  v  at  the  present  step  is  known. 
If  a  state  can  be  brought  into  M  with  that  information,  we 
shall  say  that  it  is  capturable  modulo  M  . 


We  have 


Therefore  one  of  the  two  following  possibilities  must  arise: 

•  All  geometric  differences  non-empty  W^  = 

•  Otherwise  W  =  0  . 

k 

The  condition  for  W^  to  be  non-empty  is 

Po2Qo  <W2<!1  •••  VPi2Vl 

which  can  be  transcribed  in  teims  of  the  matrices,  considering 
:t  as  the  matrix  corresponding  to  the  projection  operation 

x'ank  [ifGjrtJ]  =  rank  [kG]  rank  [rtGj  JtFGj  rtFJ] 

=  rank  [JtG;JiFG  ,  etc. 

It  suffices  that  P  =2  Q  or  rank  [rtF^G;  JtF^J]  =  rank  [«FqG]  Vq 

Q  Q  ( 

it  suffices  also  that  P  2q  or  rank  [G*J]  =  rank  [G]  . 


These  three  conditions  are  increasingly  restrictive;  the  first 

one  only  is  necessary.  Because  Jt  and  F  generally  do  not 

commute,  P  2  Q  does  not  imply  the  second  condition.  Notice 
’  o  o 

that  the  existence  of  W  implies,  as  it  should,  the  existence 
(  co) 

of  W  .  The  interesting  fact  is  that  they  are  equal. 

K 

The  conclusion  about  the  capturability  modulo  Jt  in  exactly  k 
steps  is  the  same  as  in  the  case  of  the  strong  controllability. 
Only  the  conditions  under  which  the  controllable  states  are 
capturable  are  more  stringent. 

y)  Ideal  Capturability.  No  information  is  available  on  v  .  By 
analogy  with  the  concept  of  ideal  observability,  we  shall  say  of 
a  state  that  can  be  brought  into  M  in  that  case  that  it  is 
ideally  capturable  modulo  M  .  We  have 


Therefore,  two  possibilities  again  arise: 

•  All  geometric  differences  are  non-empty 

•  Otherwise 

The  condition  for  to  exist  is 

k 

qo  =  (o)  po  +  p13q2 

and  this  condition  can  be  written  in  terms  of  the  matrices: 

Jtj  =  o  rank  [nG>nFJ]  =  rank  [nG]  rank  [ nG •  *FG j JtFqJ ] 

=  rank  [rtGjrtFG]  , 
etc. 

Again,  it  suffices  that  P  2  Q  ,  rank  [^Fq  1G;kFqJ]  = 

Q  ■■  Q 

rank  [rtFq’"1G]  V  q  but  the  first  condition  only  is  necessary. 

The  situation  is  similar  to  what  it  was  in  the  two  previous  cases, 
with  even  more  restrictive  conditions  that  clearly  imply  the  existence  of 
the  two  other  sets,  and  then  they  all  are  equal. 
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3.6  Discussion,  Optimality 


We  find  that  in  the  case  of  unbounded  controls,  unlike  in  the  other 
case,  a  change  in  the  information  structure  does  not  change  the  nature  of 
the  sets  of  controllable  states.  It  only  changes  the  condition  under 
which  these  sets  have  the  desired  property. 

In  other  words,  if  some  states  are,  say,  ideally  capturable  in  k 
steps,  then  all  states  controllable  in  k  steps  are  strongly  controllable, 
capturable  and  ideally  capturable.  What  can  be  changed  by  the  information 
structure  is  the  subspace  M  ,  modulo  which  the  system  has  the  discussed 
properties.  In  particular,  changing  the  information  may  allow  us  to 
bring  more  coordinates  of  the  state  to  zero.  This  is  not  in  contradiction 
with  our  previous  statement  which  holds  for  a  fixed  subspace  M  . 

As  far  as  the  optimality  of  the  capture  time  is  concerned,  we  have, 
of  course,  the  same  "step  by  step"  optimality  as  in  Section  3.3.  But  the 
"global"  problem  is  now  much  simpler. 

We  consider  the  relation 

x (l )  =  F  x(0)  -  G  u  (0)  +  J  v(0)  £  VR  , 

and  we  know  that  the  pursuer  can  achieve  this  if,  letting  P  =  range  G  , 

F  x(0)  +  J  v(0)  €  Vk  +  P  .  (3.6) 


As  V  +P  is  a  vector  space,  (3.6)  is  equivalent  to  a  set  of  linear  equa- 
k 

tions  on  v  .  If  x'(0)  does  not  belong  to  Vk+1  ,  then,  by  definition, 
it  is  not  verified  identically.  Then,  the  set  of  all  v’s  that  verify 
it  is  an  affine  set  in  V  ,  possibly  empty.  The  union  of  a  finite  number 
of  such  sets  cannot  be  the  whole  space  V  . 


Thus,  if  T(x)  =  p  :  x  €  V  and  x  g  V  V  k  <  p  ,  then  there  are 

P  K 

v’s  for  which  none  of  the  relations  (3.6),  with  every  k  smaller  than 
p  ,  is  verified. 

This  solves  the  problem  by  showing  that  p  is  indeed  optimal. 
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3.7  Invariant  Capture  Space 


We  are  going  to  investigate  the  case  where  the  subspace  M  is  in¬ 
variant  under  F  .  A  reason  for  doing  so  is  that  it  corresponds  to  a 
natural  problem  in  the  frame  of  modern  algebraic  system  theory.  M  is 
then  a  submodule  of  the  module  structure  induced  on  X  by  polynomials 
in  F  . 


The  results  take  a  simple  form,  and  we  are  able  to  generalize  to  the 
multiple  input  case  a  result  proved  by  Kalman  [l9]  in  the  single  input 
case. 

i)  The  Strong  Controllability  Theorem.  We  first  prove  two  simple 
lemmas . 


Lemma  1:  If  a  state  is  controllable  modulo  M  in  p  steps,  it  is  also 
controllable  in  p+q  steps,  q  >  0  .  This  is  an  immediate  conse¬ 
quence  of  the  invariance  of  M  .  Translated  in  our  notations, 
this  implies: 


itF 


'■«? 


-1  p+q-1 

pi  =*>  jtFp+qx  e 


r-. 


V  q  >  0  . 


Lemma  2:  If  W ^  is  non-empty,  then  W^  is  not  empty  either,  for 

-  p  p+q 

every  q  >  0  . 


Proof:  Assume 


This  means  that  for  every  sequence  VQ» vy» * *Vp_i  »  there  exists  a 
corresponding  sequence  u0>uy » • • ,Up-i  such 


^  «FkGuk  =  ^  »FkJvk  , 


or  equivalently 

?  "H '  t 


F  Jv.  e  M  . 
K 
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Using  the  invariance  of  M 

%  *k+K  - 

and  as  this  is  possible  for 
to 


under  F  ,  we  multiply  by  Fq 

Jv,  €  M  , 
k 

every  sequence  v^  ,  it  is  equivalent 


which  together  with  the  relation  we  started  from  gives 


or 


W 


(otj 

p+q 


X 


Pj[  *  0 


We  can  now  prove  the  following  theorem: 
Theorem:  When  M  is  invariant  under  F  ,  then 


•  Either  all  the  states  controllable  with  v  alone  (modulo  M  ), 
are  controllable  with  u  ,  and  then  all  the  states  controllable 
with  u  are  strongly  controllable; 

•  Or  no  state  is  strongly  controllable. 


Proof:  Assume  that  some  states  are  strongly  controllable.  Then  there 

(  °o)  ( 

exists  a  non-empty  W  ,  and  thus,  by  Lemma  2,  W  is  non-empty. 

k  n 


(3.7) 


By  Lemma  1,  all  states  controllable  with  u  are  given  by 


rtFnx  €  ^ 


P. 

i 


and  all  states  controllable  by  v  similarly  with  the  Q^s  .  Thus,  if 
(3.7)  is  verified,  all  states  controllable  with  v  are  controllable 
with  u  ,  and  all  states  controllable  with  u  verify 


«F  x 


£  W 


(«) 

n 
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and  thus  are  strongly  controllable. 


If  (3.7)  is  not  verified,  no  state  is  strongly  controllable,  since 
it  is  verified  as  soon  as  some  are.  This  ends  the  proof. 

ii)  Absolute  Concepts.  We  want  to  investigate  under  what  condition, 
once  the  pursuer  has  brought  the  state  in  M  ,  he  will  be  able  to  hold  it 
in  M  .  When  he  is  able  to  do  so,  we  shall  say  that  the  system  is  abso¬ 
lutely  strongly  controllable,  capturable  or  ideally  capturable.  Condi¬ 
tions  for  this  to  happen  in  the  case  where  M  is  not  invariant  can  be 
given,  but  they  are  not  very  interesting.  Here,  with  M  invariant,  the 
situation  is  very  simple. 

Let  us  first  make  a  few  remarks  about  this  question  in  the  case  of 
the  strong  controllability.  Let  Z  be  the  smallest  integer  for  which 

(oo  )  (oo  ) 

W  is  not  empty.  Then  W  contains  the  origin.  Thus,  with  the  in- 

If  If 

variance  of  M  ,  if  the  state  has  been  captured  at  time  p  ,  we  have 

x  (p)  €  M  F^x(p)  £  M  KF^x(p)  =  06  wj"* 

and  the  pursuer  is  able  to  have  the  state  return  to  M  every  Z  instants 
of  time.  However,  if  he  wants  the  state  to  belong  to  M  at  a  given  in¬ 
stant  m  larger  than  p+Z  ,  m  =  p+q  ,  he  can  always  achieve  this  since 

Fqx(p)  €  M  *Fqx(p)  =06  W^0 

(oo)  • 

and  we  know  that  W  does  exist. 

q 

If  we  want  the  system  to  be  absolutely  strongly  controllable,  then 

(oo) 

W^  must  exist: 

P  2  Q  . 
o  o 

This  insures  P  2  Q,  due  to  the  invariance  of  M  ,  as  is  easily  checked: 
k  k 

if  for  every  v  there  is  a  u  such  that 

jtGu  =  JtJv  Gu  -  Jv  €  M  , 

we  can  multiply  both  sides  of  the  second  inclusion  by  Fk  ,  which  gives 
the  result.  But  chis  implies  cnpturability.  We  therefore  have  the  fol¬ 
lowing  result: 
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Theorem:  When  M  is  invariant  under  F  ,  the  concepts  of  absolute  strong 

controllability,  capturability  and  absolute  capturability  are  equivalent. 

The  problem  of  ideal  capturability  is  of  no  interest  when  M  is  invariant, 
since  ideal  capturability  requires  that  the  range  of  J  be  in  M  ,  and 
then  the  evader  would  have  no  control  on  rtx  • 

This  finishes  our  discussion  of  the  discrete  problem. 


4.  THE  ISOTROPIC  ROCKET  GAME  AS  AN  EXAMPLE 


In  this  short  chapter,  we  present  a  special  pursuit-evasion  game: 
the  Isotropic  Rocket  Game  (I.R.G.).  Our  aim  is  to  discuss  its  formula¬ 
tion  and  to  apply  to  it  the  results  of  the  previous  theory. 

4.1  Description  of  the  I.R.G. 

The  Isotropic  Rocket  Game  was  proposed  by  Rufus  Isaacs  in  [17]  and 
[18 ].  In  these  references,  Isaacs  gave  an  analysis  which,  although 
farther  than  ours  from  being  complete,  brought  out  several  new  and  inter¬ 
esting  features.  This  analysis  covers  most  of  what  we  present  in  Sec¬ 
tions  5.2,  5.3  and  6.2.  We  shall  often  refer  to  this  work.  We  have 
tried  to  stay  as  close  as  possible  to  the  notations  of  [18],  However,  it 
was  not  always  possible  to  keep  exactly  the  same  notation,  partly  because 
this  game  appears  at  two  different  places  in  the  book,  with  different 
notations.  A  correspondence  between  ours  and  those  of  these  two  discus¬ 
sions  is  given  in  Appendix  C. 

In  this  game,  the  dynamical  possibilities  of  the  two  players  are 
as  follows: 

P.  The  pursuing  object  is  to  be  thought  of  as  a  rocket  able  to 

direct  its  thrust  in  any  direction,  whence  the  name  of  the  game. 
It  has  a  bounded  thrust- to-mass  ratio,  that  is,  an  acceleration 
the  magnitude  of  which  cannot  exceed  a  fixed  value  F  .  Within 
this  restriction,  this  acceleration  can  be  changed  instantly 
and  is  the  pursuer's  control. 

E.  The  pursued  object  is  a  maneuverable  target,  with  bounded 

velocity.  The  maximum  possible  magnitude  of  this  velocity  is 
w  .  Within  this  restriction,  it  can  be  changed  instantly  and  is 
the  evader's  control. 

Notice  that  neither  of  these  two  descriptions  is  very  realistic.  A 
rocket  is  not  steered  by  instantly  changing  the  direction  of  its  thrust, 
and  we  allow  the  target,  an  aircraft  or  an  incoming  missile,  for  instance, 
infinite  accelerations.  However,  simplified  as  it  is,  this  model  will 
still  give  meaningful  non-trivial  results  about  the  chase.  In  addition, 
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it  will  yield  new  concepts  for  a  general  theory  of  differential  games. 

Capture  is  obtained  when  the  relative  distance  of  the  two  players 
falls  below  a  fixed  radius  of  capture  i  .  This  can  represent  two  players 
of  finite  radii  i  and  i  with  f  +i  =  &  ,  or  a  pursuer  with  lethal 

X  4  1  4 

radius  &  pursuing  a  point-like  target.  We  shall  think  of  it  in  this 
second  and  more  realistic  way,  knowing  that  the  analysis  is  equally  valid 
for  the  other  case.  Whether  the  capture  set  C  must  be  regarded  as  an 
open  or  a  closed  sphere  is  not  important  at  the  modelling  stage.  Depend¬ 
ing  on  the  techniques  used,  the  question  will  be  answered  in  the  way  that 
best  fits  the  mathematical  formulation. 

4.2  Dimension  of  the  Geometrical  Space 

The  chase  occurs  in  the  three-dimensional  physical  space,  but  we 
neglect  gravity.  To  recover  Isaacs'  two-dimensional  formulation,  we  im¬ 
mediately  state  the  following  fact: 

Proposition:  An  optimal  chase  occurs  in  a  fixed  plane. 

■  y 

Proof:  Let  r  be  the  vector  from  P  (center  of  the  capture  sphere)  to 

E  ,  and  v*  be  P's  velocity.  Consider  the  plane  II  defined  at  each 
instant  by  P  ,  r  and  v  ,  which  thus  contains  E  .  Take  a  moving 
rectangular  coordinate  system  (x,y,z)  with  its  origin  at  the  point  P  , 
and  such  that  the  x-  and  y-axes  are  in  II  :  for  instance,  the  y-axis 
aligned  with  .  In  these  axes,  the  relative  coordinates  of  E  are 


and  P's  velocity  is 


This  coordinate  system  has  an  angular  velocity  o>  with  respect  to  the 
inertial  space: 


CD  =  0) 


Consider  a  new  coordinate  system  (X,Y,z)  ,  still  rectangular,  with  the 
same  origin  and  the  same  z-axis,  but  having,  with  respect  to  the  previous 
one,  an  angular  velocity  -o>z  .  This  new  system  has,  with  respect  to  the 
inertial  space,  an  angular  velocity  if  : 


*“>  — f 

and  r  and  v  have  for  components 


7-  v 


O 

Let  ^  denote  the  time  derivative  with  respect  to  the  axes  (X,Y,z)  and 
dt  the  time  derivati-ve  with  respect  to  the  inertial  space.  We  decompose 
every  vector  on  the  (X,Y,z)  axes.  For  any  vector  a*(t)  ,  we  have  by 
definition 


/a(t)\  M(t)\ 

a(t)  =  P(t)  ||  =  ^<t) 

\r(t)/  \r(  t)J 

and,  by  the  classical  laws  of  kinematics, 
da*  da*  -> 

at  =  dt  "  a  • 

Applying  this  to  r  and  noticing  that  ~  =  w  -  ?  ,  we  find 

dt  1 

*  -  *x  -  u 
Y  =  wy  -  V 

0  =  wz  +  Xciiy  -  Yo>x 


and  similarly  with  77  =  ?,  which  gives 
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0  =  F£  +  Uoy  -  VtOj.  . 

2  2  2 

If  wo  notice  that  capture  is  defined  by  X“+Y  <  £  ,  we  see  that  all  the 

information  is  contained  in  the  four- dimensional  gome  in  X,Y, U,V  .  For 
this  game,  the  controls  are  the  projections  (w^, w  )  and  (F„, F^r)  of 
w  and  F  on  Jl  . 

Moreover,  the  dynamical  equations  of  that  gome  are  linear,  of  type 
(1.1).  We  can  apply  the  optimality  principle,  derived  in  Chapter  Two, 
and  we  find  that  the  optimal  strategies  must  verify 

2  2  2  2  2  2 
w  +  v  =  w  F*  +  Fy  =  F 

and  therefore 

w  =  0  F  =  0  . 

z  z 

Placing  this  in  the  z  equations  of  our  two  differential  systems  yields: 

COj.  =  CDy  =  0  . 

(When  r  and  v  are  aligned,  we  can  choose  this  solution.) 

Therefore,  the  coordinate  system  (X,Y,z)  has  a  fixed  direction  in 
inertial  space.  As  a  consequence,  the  plane  II  ,  in  which  the  chase 
occurs,  con  be  considered  ns  fixed  in  space. 

This  proves  the  proposition, 

*1 . 3  Representations 

Two  main  representations  of  our  dynamics  will  be  used. 

i)  4-D Represen to tions .  The  first  one  is  four-dimensional.  The 
origin  of  the  coordinate  system  is  at  the  centor  of  the  pursuer's  circle 
of  capture.  The  oriontation  of  the  axes  is  fixed  in  inertial  space.  The 
state  variables  are  the  relative  coordinates  X,Y  of  the  evader,  and  the 
components  U,V  of  the  pursuer's  velocity. 


GO 


The  equations  of  motion  are 


with 


X  =  -U  +  w 
Y  =  -V  +  w 


2  2  2 
WX  +  wy  - 


•> 

F“  +  F 


(4.1) 


This  form  is  linear,  and  the  previous  theory  will  apply  to  it  directly. 
Notice  a  vectorial  formulation  of  it,  with  vectors  of  the  geometrical 
space: 


where  &  and  0  are  unit  vectors,  the  direction  of  which  are  the  con¬ 
trols.  Here  it  has  already  been  assumed  that  the  playex-s  choose  their 
controls  on  the  boundary  of  the  conti-ol  sets. 

ii)  3-D  Representations.  It  is  possible  to  find  a  three-dimen¬ 
sional  representation:  its  equations  are  much  more  complicated  than  (-1.1), 
and  non-linear,  but  it  will  be  desirable  in  the  subsequent  theory  to  use 
the  lowest  dimensional  representation. 

The  game  is  obviously  insensitive  to  absolute  orientation  in  the 
plane.  We  can  take  advantage  of  this  by  choosing  the  y-axis,  for  instance, 
parallel  to  the  pursuer's  velocity.  Then  this  velocity  is  represented  by 
a  single  variable,  its  magnitude. 

Again,  we  assume  that  both  players  choose  controls  of  maximum  mag¬ 
nitude,  so  that  we  can  represent  their  controls  by  a  single  parameter  fox- 
each.  Following  Isaacs,  wo  choose  to  give  the  directions  of  these  con¬ 
trols  by  theix-  angle,  measured  clockwise  from  the  y-axis:  <p  for  the 
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pursuer  and  for  the  evader. 

The  equations  of  motion  are  easy  to  derive.  They  are  (see  Appen¬ 
dix  A): 

*  Py 

x  =  -  —  sin  cp  +  w  sin 
v  T 

y  =  —  sin  <p  +  w  cos  -  v  (4.3) 

v  =  F  cos  <p  . 

These  coordinates  are  related  to  the  previous  ones  through  the  formulas 
v  =  JlFrf 

x  =  -  (-UY  +  VX)  (4.4) 

v 

y  =  —  (UX  +  VY)  . 
v 

In  this  system,  the  capture  set  is  a  cylinder  of  revolution  around  the 
v-axis.  As  a  consequence,  we  shall  also  use  the  cylindrical  form  of  the 
same  coordinates: 

x  =  r  sin  9 
y  =  r  cos  Q 

and  the  equations  of  motion  now  are 

r  =  w  cos  (ty-0)  -  v  cos  9 

9  =  -  —  sin  a)  +  —  sin  (t-0)  +  —  sin  9  (4.5) 

v  r  r 

v  =  F  cos  9  . 

The  various  coordinate  systems  are  depicted  in  Fig.  2. 

iii)  Parameter.  The  unit  of  length  can  be  chosen  arbitrarily,  so 
as  to  assign  any  desired  numerical  value  to  l  .  This  being  done,  the 
unit  of  time  can  still  be  chosen  so  as  to  assign  any  desired  value  to  w 
Then  the  game  is  completely  defined  by  a  single  numerical  value  for  F  . 
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In  our  analysis,  we  have  chosen  not  to  nondimensionalize,  in  order  to 
let  the  nature  and  meaning  of  intermediary  quantities  and  relations  be 
more  apparent.  But  the  previous  remark  shows  that  a  single  non-dimen¬ 
sional  parameter  is  needed  to  characterize  the  game.  We  shall  use 

2 

w 

P  “  2F£  * 

The  factor  two  in  the  denominator  has  been  put  there  for  reasons  of  con¬ 
venience  that  will  appear  later. 


4.4  Results  From  the  Previous  Theory 

i)  Formulation.  We  use  the  linear  representation,  and  we  define: 


And  the  matrix  C  is  then 


C  = 


/0  0  1  o\ 
/  0  0  0  1  \ 
0  0  0  0 
\0  0  0  0/ 


eTC  =  0(t) 


/l  0  t  0\ 

/  0  1  0  T 

1  0  0  1  O’/ 

\0  0  0  1/ 


P  and  Q  are  disks  in  their  respective  subspaces.  The  geometrical  sub¬ 
space  is  the  subspace  of  the  first  two  coordinates,  in  which  capture  is 
defined  by 

C  =  (zlx2  +  ^  <  /} 

where,  to  comply  with  the  formulation  of  our  theory,  C  has  been  chosen 
as  a  closed  set. 

The  operator  kQ(t)  is  given  by  the  matrix 

-  (i ;  s  °) 

so  that  P  and  Q  are  circles  centered  at  the  origin  and  of  respec- 
T  T 

tive  radii  tF  and  w  .  Notice  that  whatever  the  relative  value  of  the 
parameters,  for  small  enough  T  ,  P^  C  . 
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It  is  shown  in  Pontryagin  [24]  that  this  implies  that,  if  the  evader 
knows  the  present  control  of  the  pursuer,  capture  is  impossible  if  it  is 
defined  as  point  coincidence:  l  =  0  .  We  shall  reach  this  same  conclu¬ 
sion  without  the  assumption  that  the  pursuer's  control  is  known  by  the 
evader  (see  Chapter  5).  In  fact,  we  are  in  a  case  where  the  sets  W 

T 

are  disks,  and  have  a  single  normal  at  each  point  of  their  boundary. 
Therefore,  the  concluding  remark  of  Section  2.5  holds;  (2.7)  and  (2.7a) 
define  unambiguously  the  controls  u*  and  v*  ,  independently  of  each 
other. 

ii)  Estimating  Function.  The  sets  C  ,  and  are  all 

disks.  After  Pontryagin,  we  notice  that  in  this  case  the  operations  of 
sum,  geometric  difference  and  integral  of  sets  reduce  to  sum,  difference 
and  integral  of  radii.  The  alternating  integral 


W 

T 


4  Q  ]dr 
r 


is  the  disk  centered  at  the  origin,  and  of  radius  Q(r)  given  by 


Q(t)  =  £  +  J  (rF-w)dr  =  F  •—  -  wt  +  Z  .  (4.6) 

and  WT  exists  as  long  as  Q(r)  is  non-negative  for  every  t  smaller 
o 

than  tq  .  £  is  given  by 

iM  ■  «»<T)Z  -  O 

so  that  T(z)  =  t  is  the  smallest  positive  root  of  the  equation: 
o 

(X-tU)2  +  (Y-tV)2  =  Q(x)2  .  (4.7) 

We  can  use  the  formulation  (4.2)  to  express  (4.7)  in  a  different  form 


£  =  r  -  tv 

so  that  (4.7)  becomes 

||?  -  tv]|  =  Q(t)  .  (4.7a) 

We  notice  that  Q(t)  is  quadratic  in  t  ,  when  ||£(t)||  is  linear.  Thus 
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equation  (4.7a)  always  has  a  solution  if  ||£(0)||  >  Q(0)  ,  namely 
||r]|  >  i  ,  which  is  always  verified  for  the  starting  point. 

This  means  that  if  Q(t)  does  not  vanish,  capture  will  always  occur 
in  finite  time.  Q(t)  never  vanishes  if  its  determinant  is  negative: 

2 

w  -  2Fi  <  0  equivalently  p  <  1  . 

We  therefore  have  the  following  important  result: 

Proposition:  For  p  smaller  than  one,  capture  occurs  from  all  initial 
conditions . 

iii)  Barrier.  We  have  a  somewhat  simpler  way  of  using  equation 
(4.7a).  Instead  of  translating  r  by  -vt  ,  we  prefer  to  translate  W^ 
by  +vt  and  directly  check  whether  the  point  considered  belongs  to  this 
capture  set. 

Drawing  these  sets  for  a  given  v  ,  we  obtain  Fig.  3,  which  is  the 
same  as  Fig.  5.5.4,  p.  114,  in  [18 3,  although  obtained  by  completely  dif¬ 
ferent  means. 

The  most  prominent  feature  of  this  figure  is  the  existence  of  an 
envelope.  It  is  a  line  of  non-regular  points  of  first  kind,  or  barrier. 
Its  equation  is  easy  to  establish.  We  use  the  axes  of  the  three-dimen¬ 
sional  representation.  Then  the  circles  verify  the  equation: 

2  2/12 
x  +  (y-v-r)  =  I  —  Ft  -  WT  +  f 

and  their  envelope  is  given  parametrically  by 


The  double  sign  in  x  accounts  for  the  two  symmetric  parts  of  the  en¬ 
velope.  Here,  the  parameter  t  is  the  estimated  time  to  go  just  inside 
of  the  discontinuity,  and  on  the  barrier  itself,  which  verifies  the  con¬ 
ditions  of  the  last  theorem  of  Section  2.11.  (But  the  trajectories  of 


72 


the  barrier  do  not  have  the  shape  of  the  envelope  (4.8),  since  they  have 
a  varying  v  .) 

When  p  >  1  ,  Q(t)  vanishes  for  some  ,  this  envelope  defines  a 
curvilinear  triangle  of  increasing  size  as  v  is  increased.  The  locus 
of  its  vertex  on  the  y-axis  is  a  straight  line  at: 

x  =  0 

y  =  vtx  . 

We  shall  later  refer  to  this  line  as  the  "crest."  We  are  insured  that, 
inside  this  region,  capture  will  always  occur. 

4.5  Conclusion 

The  technique  developed  in  the  first  part  has  given  us  a  positive 
answer  to  the  problem  of  completion:  for  p  <  1  capture  is  always  pos¬ 
sible.  Moreover,  if  we  can  check  that  the  trajectories  do  not  cross  tha 
barrier  or  penetrate  the  capture  circle,  we  have  the  optimal  time  of  cap¬ 
ture  and  the  optimal  strategies.  This  will  be  found  to  be  the  case  for  a 
large  region  of  the  state  space. 

However,  it  will  be  seen  that  some  of  these  trajectories  would  in 
fact  penetrate  C  .  Consequently,  we  do  not  have  the  optimal  strategies 
for  the  region  these  trajectories  come  from.  Neither  can  we  assert 
that  for  p  >  1  evasion  occurs  from  outside  the  region  of  finite  T(z)  . 

This  problem  will  be  investigated  in  the  next  chapter  by  trying  to 
construct  directly  all  the  barriers.  It  will  be  seen  that  escape  is 
probably  not  possible  unless  p  is  larger  than  some  pq  larger  than  one. 


5.  THE  GAME  OF  KIND 


In  this  chapter,  we  generalize  the  concept  of  barrier  and  use  it  to 
investigate  the  game  of  kind,  the  outcome  of  which  is  qualitative: 
capture  or  escape. 

5.1  Semi-Permeable  Surfaces 

i)  Analytical  Description.  In  [18],  Isaacs  introduces  the  concept 
of  semi-permeable  surfaces.  Let  the  dynamics  of  a  game  be  (we  use  cp 
and  for  the  controls): 

z  =  f(z,<p,\!0  . 

Let  S  be  a  surface  and  v  Its  normal.  Suppose  it  is  such  that 

min  max  (v,f  (z,<p,ty))  =  0  .  (5.1) 

9  V 

We  shall  always  assume  that  f  is  "separated,"  that  is,  of  the  form 
f(z,<p,ijO  =  h(z,t)  -  g(z ,q>) 

so  that 

min  max  (v,f (z,q>,i|0)  =  max  (v,h(z,\|0)  -  max  (v,g(z,<p))  . 
cp  *  V  cp 

Equation  (5.1)  has  an  obvious  geometrical  meaning:  it  states  that  player 
E  cannot  force  the  state  to  cross  S  in  the  direction  of  v  ,  when 
player  P  cannot  force  it  to  cross  S  in  the  other  direction.  Thus, 
if  they  both  try  to  do  so,  the  ensuing  motion  will  be  in  S  .  S  is  called 
a  semi-permeable  surface.  Its  analogy  with  the  barrier  of  Chapter  Two 
is  obvious,  particularly  in  view  of  the  last  theorem  of  that  chapter. 

In  particular,  if  such  a  surface  defines  a  closed  region  containing 
the  capture  set,  with  v  pointing  outside,  the  evader  can  make  sure  he 
will  never  penetrate  this  region  if  he  starts  from  outside,  and  thus  never 
be  captured. 

Even  if  this  region  is  open,  such  a  surface  can  still  represent  a 
discontinuity  of  the  capture  time  if  capture  is  only  possible  on  one  side 
of  it. 
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The  way  to  construct  such  surfaces  is  discussed  in  [18];  we  shall 
indicate  it  only  briefly. 

Given  a  line  of  initial  conditions,  one  first  determines  at  each 
point  of  it  a  vector  v  normal  to  this  line  and  verifying  (5.1).  The 
same  relation  also  determines  <p  and  ^  ,  uniquely  if  the  "vectorgrams" 
P  and  Q  are  strictly  convex,  and  thus  z  . 

The  differential  equations  for  the  normal  v  along  a  trajectory 
are  well  known  to  be  the  adjoint  equations.  See,  for  instance,  [3],  Sz 
being  a  vector  tangent  to  the  surface,  it  verifies 


and  taking 


where  the  star  denotes  the  adjoint  operator,  we  have,  defining  q  as 


q  =  \V,5z) 


so  that  if  q  is  zero  at  some  time,  it  is  at  every  time,  and  we  check 
that  v  stays' normal  to  the  surface. 

ii)  Geometrical  Description.  Notice  that  (5.1)  provides  one  rela¬ 
tion  only  between  z  and  v  »  so  that  at  each  point  of  the  state  space, 
there  usually  exists  a  cone  (hypercone)  of  "semi-permeable  v’s  ,"  and 
a  corresponding  cone  of  "semi-permeable  directions"  f  .  We  propose  a 
simple  geometric  construction  of  these  two  cones. 

For  a  given  point  z  ,  let  P  =  g(z,tp)  ,  9  6  $  the  set  of  allow¬ 
able  9's  ,  and  let  Q  =  h(z,^)  ,  ^  €  ¥  the  set  of  allowable  Y's  . 
Notice  that  as  compared  to  our  earlier  definitions,  the  terms  independent 
of  the  controls  in  f  ,  Cz  ,  for  instance,  have  been  arbitrarily  cast 
into  one  of  the  functions  h  or  g  ,  thus  translating  P  or  Q  by  the 
same  amount. 
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Let  a  bitangent  plane  be  a  plane  (hyperplane)  H  such  that 

1)  II  contains  one  point  at  least  of  each  of  the  sets  P  and  Q  ; 

2)  P  and  Q  are  both  entirely  contained  in  the  same  half  space 
defined  by  II  . 

Let  v  be  normal  to  II  ,  opposite  to  the  half  space  containing  P 
and  Q  . 

Proposition:  v  is  a  semi-permeable  normal.  The  corresponding  semi-per¬ 
meable  directions  are  the  vectors  joining  any  point  af  uHp  to  any  point  of  n(~)Q  . 

Proof:  Let 

g(z ,cp*)  enHp  h(z,f*)  enriQ  . 

Because  of  property  two 

(v,g(z,<p))  <  (v,g(z,c p*))  V  cp  €  ® 

<V,h(z,V))  <  (v,h(z,^*))  V  \Jr  €  I  . 

Thus 

<V,f(z,<p*,t*))  =  min  max  (v,f  (z,<p,f) )  . 
cp  \|r 

Since  g(z,cp*)  and  h(z,ty*)  both  belong  to  II  ,  their  difference  is 
parallel  to  II  ,  hence  normal  to  v  .  Therefore,  relation  (5.1)  is  veri¬ 
fied  and  the  proposition  proved  (see  Fig.  3). 

iii)  The  I.R.G.  In  our  case,  with  f  given  by  (4.3),  we  can 
represent  the  vectorgram  as  follows  (see  Fig.  4). 

In  an  (x,y,v)  space,  visualized  with  its  axes  parallel  to  the 
(x,y,v)  axes,  Q  is  a  circle  of  radius  w  centered  at  the  origin  and 
lying  in  the  xy  plane;  P  is  an  ellipse  centered  at  a  point  (0,v,0) 
with  one  principal  semi-axis  of  length  F  parallel  to  the  v  axis,  and 
the  other  one,  of  length  —  in  t:.e  xy  plane  normal  to  r  . 

In  the  case  drawn  in  Fig.  4,  there  are  two  separate  cones  of  v's 
given  by  our  construction,  one  "above"  the  plane  of  P  and  one  "under." 
Correspondingly,  there  are  two  cones  of  semi-permeable  directions. 
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If  one  extremity  of  the  axis  of  P  lying  in  the  xy  plane  is  in¬ 
side  Q  ,  then  there  is  a  single  continuous  family  of  v's  ,  and  of  semi- 
permeable  directions.  By  elementary  geometry,  it  is  easy  to  see  that  we 
are  in  the  first  case  for 

2  2 

F  r  2  2 

Q  =  — —  +  v  -  2Fx  -  w  >  0  . 

JL 

V 

If  <  0  ,  the  two  cones  merge  together  to  yield  the  second  case.  This 
family  splits  again  into  two  separate  cones,  on  each  side  of  the  xy 
plane,  when 

2  _  2 

Q0  =  +  v  +  2Fx  -  w  <  0  . 

v“ 

Notice  that  for  positive  x  (the  game  being  symmetric  with  respect  to 
the  yv  plane;  we  shall  always  consider  this  half  space),  Q  has  a 

2  2  “ 

minimum  for  v  =  Fr  ,  x  =  0  ,  y  =  £■  ,  Q9  =  2Ff-w  ,  so  that  it  can 
be  negative  only  when  p  >  1  . 

Notice  that  for  or  Q9  equal  to  zero,  a  particular  semi-per¬ 

meable  direction  is  f  =  0  ,  meaning  that  relative  rest  satisfies  (5.1). 
The  sign  of  will  turn  out  to  be  important  in  part  of  the  analysis. 

5.2  The  Natural  Barrier 

i)  The  B.U.P.  It  is  pointed  out  in  [18]  that  the  game  can  terminate 
only  in  the  "usable  part"  of  the  capture  set,  such  that,  v  being  the 
outward  normal, 

min  max  {v,f  (z,<p«V))  <  0  • 

9  V 

For  convenience,  the  capture  set  will  now  be  considered  as  open,  so  that 
trajectories  arriving  tangentially  to  it  still  provide  escape.  Hence 
the  strict  inequality. 

This  usable  part  has  a  boundary  given  by 

min  max  (v,  f  (z,<P,V))  =  0  • 

<p  V 


Comparing  this  with  equation  (5.1),  it  is  clear  that  we  can  attach  to  this 
line  a  semi-permeable  surface  having  the  same  normal  v  ,  and  thus  tan¬ 
gent  to  C  .  Trajectories  of  this  surface  do  not  provide  capture  along 
the  B.U.P. 

This  surface  locally  separates  the  state  space  into  two  regions. 

The  first  one  contains  the  usable  part  of  C  ,  and  a  game  starting  from  a 
point  of  this  region  can  be  completed  in  a  simple  way.  But  from  a  point 
in  the  other  region,  if  capture  is  possible  the  trajectory  must  in  some 
sense  go  around  the  surface.  Therefore,  this  surface  is  a  barrier;  it 
represents  a  discontinuity  in  the  time  of  capture.  This  barrier  emanating 
from  the  B.U.P.  is  called  the  natural  barrier. 

In  our  case,  the  boundary  of  the  usable  part  is  the  curve  $  defined 
by 

min  max  [w(x  sin  <p  +  y  cos  (p)  -  vy]  =  wi  -  vy  =  0 
cp  ♦ 


or 

w  -  v  cos  0=0. 

It  exists  only  for  v  >  w  .  Projected  on  the  yv  plane,  it  appears  as 
a  hyperbola,  extending  from  v  =  w  ,  y  =  Jl  to  infinity  asymptotic  to 
the  v-axis. 

ii)  Equations  of  the  Natural  oai'r'°r.  At  this  point,  we  need  to 
establish  the  differential  equations  resulting  from  (4.3)  together  with 
(5.1). 

(5.1)  gives,  with  the  components  of  v  being  vx  ,  vy  and  vy 


Introduce  the  following  notations: 
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V.  = 


a  = 


yv  -  xv 
x  y 


nr 

h 

2 

+  V 

i  2 

1  V 

V 

=  vAg  +  v2vJ 


The  strategies  satisfying  the  minimax  condition  are  given  by  (5.3),  which 
shows  that  the  evader  points  his  velocity  parallel  to,  and  in  the  same 


direction  as,  (v  ,v  ) 
x  y 


*  9 

sin  ®  =  — 

Y  va 


*  V 

cos  cp*  =  — 


,  *  X 

sin  =  — 
P 


,  *  V 

cos 


(5.3) 


and  (5.2)  becomes 


H  =  -Fa  +  pw  -  v  v  =  0  . 

1  y 


(5.2a) 


The  dynamical  equations  and  the  adjoint  equations  are  then 


yv„  V 

•  „  9  x 

x  =  -F  — r—  +  w  — 

2  p 

v  a 


v  v 

v  =-F-Ae 

x  2 

v  a 


xv.  v  v  v 

y  =  F  ——  +  w-^-v  v  =  F  ■  *-  $ 
v2a  P  y  v2a 


(5.4) 


v  =  -F 


_  9 

V  =  -F  — r—  +  v 
v  3 ,  y 

v  a  J 


These  equations  will  appear  again;  they  actually  are  the  Euler- La  grange 
equations  of  the  optimization  problem.  They  are  discussed  in  Appendix  A. 
It  is  shown,  in  particular,  that  in  the  other  coordinate  system,  their 
closed  form  integration,  with  any  initial  conditions,  presents  only  ele¬ 
mentary  difficulties. 

According  to  (5.2a),  must  be  a  first  integral  of  (5.4);  this 

can  be  checked  directly  from  the  equations.  actually  is  the  Hamil¬ 

tonian  of  the  "abnormal"  optimization  problem  in  the  game  of  degree,  as 
we  shall  see  later. 
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We  know  the  vector  v  at  the  terminal  point  of  the  corresponding 
trajectories.  Consequently,  following  the  usual  practice  of  dynamic 
programming,  we  integrate  (5,4)  backwards  from  the  curve  $  ,  calling  r 
the  time  to  go.  We  choose  the  velocity  as  the  parameter  of  SB  ,  and  to 
distinguish  it  from  the  running  variable,  call  it  s  .  Similarly,  when 
needed,  the  corresponding  angle  Q  will  be  labelled  (3  .  On  $  we  have 


x  =  SL  s/\-  (w/s)2 


V  =  p  vl~  (w/sj 
X 


y  =  i(w/s) 


Vy  =  p(w/s) 


v  =  s  V  =  0 

V 

where  p  is  an  arbitrary  parameter,  since  the  length  of  the  vector  v 
is  of  no  importance.  Notice  that  an  immediate  consequence  of  equations 
(5.4)  is  the  first  integral 

p  =  constant. 

The  solution  is,  for  the  half  space  x  >  0  , 

^2-w2  /  l  „  2  „\  J s2-w2  s 

x  _  - - —  Ft  -wt+i J  =  - - —  Q(t) 


1  fl  „2  3  1  2  . 

=  -  -Ft  -  -  Fwt  +  ( 
v  2  2 


2  2  2 

Fwt  +  (s  -w  -F^)t  +  w £ 


Q(t)  +  VT 


v  =  v/f2t2-2wFt+s2  =  \/  (w-Ft)2+s2-w2 


(5.5) 


and  for  the  adjoints 


(5.6) 


The  integrand  in  v  has  a  minimum  for  t  =  w/F  ,  and  for  this  value  our 


formula  gives 


2 

v 


2 

s 


2 

w 


>  0 


so  that  v  is  well-defined,  as  well  as  x  and  y  . 

Notice  that  x  has  the  sign  of  Q(t)  .  As  in  the  previous  theory, 
the  barrier  closes  in  the  (y,v)  plane  if  and  only  if  Q  vanishes.  And 
if  Q(t^)  =  0  ,  then  our  surface  intersects  the  (y,v)  plane  along  the 
straight  line  (3  :  y  =  vi^  .  For  p  =  1  ,  the  surface  is  just  tangent  to 
the  symmetry  plane,  and  thus  to  the  symmetric  part,  along  that  line,  with 
T1  =  w/F  . 

It  is  interesting  to  compute  the  equations  of  a  cross-section  of 
this  surface  by  a  plane  v  =  constant.  We  eliminate  s  between  v  and 
x  ,  and  obtain: 


y  = 


w-Ft 

v 


Q(t)  +  VT  . 


We  recognize  equations  (4.8),  thus  completely  identifying  this  surface 
with  the  barrier  already  mentioned. 

iii)  Termination.  However,  although  the  trajectories  (5.5)  are 
smooth,  sections  at  constant  v  of  the  surface  (4.8)  have  a  cusp  for 


value  which  is  always  larger  than  w/F  ,  and  thus  than  the  lower  root  of 
Q(t)  if  there  is  one,  so  that  this  cusp  does  not  appear  when  the  barrier 
closes. 


The  explanation  of  this  cusp  when  the  trajectories  show  no  such 
anomaly  is  that  the  surface  actually  has  a  cusp,  but  the  constituting 
trajectories  are  tangent  to  it,  so  that  they  are  smooth  across  it.  We 
have  a  verification  of  this  fact  by  calculating  the  envelopes  of  the 
pi'ojections  of  the  trajectories  on  two  different  planes,  and  checking 
that  contact  with  the  envelope  is  obtained  for  the  same  t  in  the  two 
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projections,  so  that  the  two  envelopes  are  the  projections  of  the  same 
curve  in  the  three-dimensional  space. 

It  is  lengthy  but  straightforward  to  see  that  formulas  (5.5)  give 
dx  Sv  dv  dx  Ft-w  J  2  „„  2  2 

57  3?  -  5?  57  =  s  rr T  F  T  * 2F,T " 2(s  'w  ‘w> 

2v  vs^-w^  L  J 

dy  bv  by  dv  s  _2  2  0_  _ ,  2  2  . 

57  37  ‘  57  77  =  “  F  T  "  2FwT  "  2(8  ~w  ~F^  ' 

2v  L  J 

which  agrees  with  what  we  have  just  said. 

This  phenomenon  is  interesting  in  several  respects.  First,  it  shows 
that  the  barrier  comes  to  an  end.  In  fact,  after  the  cusp,  the  surface 
is  still  semi-permeable,  but  with  the  vector  v  pointing  inside  the 
capture  region,  so  that  it  would  correspond  to  a  situation  where  the 
evader  would  be  trying  to  force  capture,  against  the  will  of  the  pursuer. 
This  part  must  thus  be  discarded. 

But  also,  it  will  appear  that  this  is  not  an  isolated  case,  but  hap¬ 
pens  on  most  of  our  barriers.  This  case  is  the  only  one  for  which  we 
have  simple  analytical  formulas  allowing  a  detailed  analysis  of  the  situ¬ 
ation.  A  full  understanding  of  the  geometry  of  this  case  will  help  in 
other  instances. 

5.3  The  Envelope  Barrier 

i)  The  Envelope  Barrier.  Another  problem  was  pointed  out  by  Isaacs. 
On  S  we  have 

9  2 

JLr  =  s’"  -  w  -  Ff  , 

so  that  for  s  <  \/w2+F i  the  trajectories  (5.4)  actually  arrive  at  $ 
from  inside  the  capture  circle,  which  they  have  thus  penetrated  at  an 
earlier  time.  This  is  a  typical  occurrence  of  the  problem  pointed  out 
in  Chapter  Two.  As  a  consequence,  these  trajectories  cannot  be  retained 
as  escape  trajectories. 

We  shall  therefore  consider  S  as  interrupted  at  the  point  B  : 
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V  =  s 


=  v^2 


+Fi 


cos  0  =  cos  {3  = 


y^a+Fi 


r  s  J 


and  the  crest  at  a  point  A' 


x  =  0  y  =  tqv  v  =  Vw^-Fi  . 

The  barrier  presents  a  "hole"  at  its  lower  v  end,  and  what  happens  in 
that  region  is  unanswered  by  the  previous  theory. 

The  way  out  of  this  difficulty  was  found  by  Isaacs:  from  B  ,  one 
constructs  a  semi-permeable  line  of  the  lower  dimensional  game  in  which 
the  state  is  constrained  to  remain  on  the  capture  circle.  It  is  shown  in 
[18 ]  that  this  line  has  the  following  properties.  Let  3)  be  this  line. 

It  is  tangent  to  S  at  B  ; 

It  is  such  that  a  barrier  can  be  constructed  from  it,  made  of  tra¬ 
jectories  that  reach  C  tangentially  to  .3)  ; 

This  barrier,  the  "envelope  barrier"  S  ,  provides  a  smooth  exten¬ 
sion  to  the  natural  barrier. 

These  facts  can  be  understood  in  the  following  way.  We  know  that  at 
each  point  of  the  state  space,  there  is  a  cone  of  possible  semi-permeable 
directions.  Taking  a  point  on  the  non-usable  part  of  the  capture  cylinder, 
we  can  find  in  this  cone  a  direction  (actually  two)  which  is  tangent  to 
the  cylinder.  This  defines  a  field  of  directions  on  the  surface  of  the 
cylinder,  equivalent  to  a  differential  equation.  The  curve  3)  is  the 
integral  of  this  equation  through  B  . 

Clearly,  v  is  normal  to  3)  at  each  of  its  points.  Therefore,  the 
trajectories  constructed  with  this  v  form  a  barrier.  By  construction, 
they  are  tangent  to  5)  ,  and  this  is  the  only  way  in  which  a  barrier  can 
reach  the  non-usable  part  of  C  without  penetrating  it. 

It  is  clear  that  once  5)  is  reached,  playing  the  strategies  of  the 
semi-permeable  surface  will  cause  the  state  to  follow  5)  since  they 
define  a  direction  always  tangent  to  it.  Moreover,  if  we  look  a  priori 
for  trajectories  straying  on  the  surface  of  C  ,  5)  appears  as  a  semi- 
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permeable  line,  since  S  itself  is  semi-permeable. 

Notice  that  !D  is  for  the  envelope  barrier  a  cusp  of  the  type  dis¬ 
cussed  in  the  previous  section,  and  the  prolongation  of  the  trajectories 
was,  of  course,  discarded. 

ii)  The  I.R.G.  With  our  equations  (4.5),  the  condition  that  a 
trajectory  lie  on  C  is 

r  =  w  cos  (ty-0)  -  v  cos  0=0 

cos  (ty-0)  =  —  cos  0 
w 

sin  (ty-0)  =  —  v4v^-v2cos20  , 
vv 


the  sign  of  sin  (ty-0)  being  chosen  in  such  a  way  that  the  evader  runs 
away  from  the  center  line,  toward  the  non-usable  part. 

The  dynamics  become: 


0  - - sin  cp  +  —  (v  sin  0  +  \/w2-v2cos20) 

V  X/ 

v  =  F  cos  <p  . 


(5.7) 


Referring  to  our  vectorgram  (Fig.  4),  the  requirement  r  =  0  obliges  the 
evader  to  choose  his  control  at  the  point  of  Q  which  lies  in  the  plane 
of  P  .  Then,  considering  the  restricted  vectorgram  in  this  plane,  our 
geometrical  theory  shows  that  the  pursuer  must  choose  his  control  at  the 
point  of  contact  of  one  of  the  two  possible  tangent  vectors.  We  choose 
the  one  that  gives  an  increasing  v  to  be  in  agreement  with  the  natural 
barrier  at  B  .  It  is  found  by  maximizing  the  ratio  v/0  ,  Notice  that 
it  is  clear  from  the  geometry  of  the  vectorgram  (Fig.  4)  that  such  tan¬ 
gents  exist  only  if  <  0  . 

.We  introduce,  following  Isaacs, 


c  =  i  (v  sin  0  +  %/w2-v2cos20)  . 

A t 


The  semi-permeable  direction  is  obtained  for 


sin  ffl  =  — 
T  c 


9  =  ±  —  %/ c2-a2 
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and  the  corresponding  equations  of  motion  are 


2  2 


This  equation  does  not  seem  to  be  integrable  in  closed  form.  Since  in 
[18]  only  analytical  solutions  are  sought,  deliberately  excluding  numeri¬ 
cal  integrations,  the  problem  is  left  at  this  point  with  the  conjecture 
that  2)  might  reach  the  (y,v)  plane,  whenever  p  >  1  at  least,  and 
the  envelope  barrier  together  with  the  natural  barrier  seal  off  a  capture 
region.  Escape  would  occur  for  any  starting  point  outside  this  region. 

It  turns  out  that  some  more  analytical  results  can  be  obtained. 

Then,  the  use  of  high-speed  computers  allows  us  to  check  them,  and  to 
proceed  further  with  the  investigation  of  the  problem  through  numerical 
integration  of  the  equations. 


5.4  Termination  of  the  Envelope  Barrier 

i)  Termination  of  3)  .  The  curve  ID  is  obtained  by  integration  of 

(5.8)  from  B  toward  lower  v's  .  This  integration  can  be  carried  out, 

2  2 

at  least  in  principle,  as  long  as  c  -  a  >  0  .  The  question  is  whether 

2  2 

it  reaches  the  symmetry  plane  before  reaching  the  surface  c  =  a  .  It 
cannot  reach  the  plane  v  =  0  where  c  is  finite  and  a  infinite. 

In  the  region  of  interest,  both  c  and  a  are  positive.  Thus,  we 

want 

c  >  a  or  v  sin  9  +  s/w^-v^cos^g  >  ££  . 

In  the  vicinity  of  9=0,  — —  v  sin  9  is  positive.  We  isolate  the 
square  root  and  square  both  sides.  This  gives 
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F2i2  2  2 

Q1  =  — —  +  v  -  2Ff  sin  0  -  w  <  0  , 

v 

which  is  consistent  with  our  remark  of  the  previous  section,  based  upon 
the  geometry  of  the  vectorgram. 

Let  Q  be  the  curve  Q^=0,  r=i.  It  has  a  minimum  in  x  for 


v  =  J. Fi 


sin  0  =  I  -  —  =  i  -  p 


From  this,  we  immediately  conclude  that 


1) 

For 

p  <  1  ,  the 

2) 

For 

P  =  1  ,  if 

v  = 

Jn  . 

Thus,  let  us  see  in  more  detail  what  happens  at  that  point. 

Equation  (5.8)  shows  that  upon  reaching  0  ,  $  has  to  be  parallel 

to  the  v-axis.  Thus,  an  integral  can  pass  through  A  only  if  it  has  a 
curvature  greater  than  that  of  ®  at  the  same  point. 


Using  its  equation  =  0  ,  we  find  for  Q  , 

2 


d20 

dv2 


sin  0 


Q 


2 .2  3 

F  i  cos  0 


v  - 


fV 


Ff  cos  0 


1  +  3 


fV 


At  A  ,  sin  0=0,  v  =  ,  we  obtain 


A 


JL 

F£  * 


‘dv  /q 

Differentiating  (5.8)  with  respect  to  0  ,  using  c  =  a  ,  one  can  check, 
after  some  rearrangements,  that 


(is.)  =  — JL— - 

Vdv/j)  F  v/v^-v^cos^©  L 

Ln  0  "j 

;3_a2J 


v  cos  0  2/  *  «  .  cf\ 

- - -  +  v  c“—a“  (sin  0  +  — ) 


2  sin 

y  c~ 


2  2 


When  v  goes  to  ,/F.g  ,  and  0  to  zero  simultaneously  with  c  -a  , 
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sin  Q  is  positive, 

± _  _  J_ 

F^  %/w2-v2cos2@ 

Therefore,  no  curve  satisfying  (5.8)  can  exist  at  A  ,  and  we  have  the 
following  result: 

Proposition:  For  p  =  1  ,  the  curve  5)  does  not  reach  the  symmetry 
plane. 

It  has  been  found  by  numerical  integration  that  it  terminates  at  a 
point  D  ,  which  is  an  equilibrium  point  of  the  relative  motion,  given  by 

v  =  0.6002  X  w  Q  =  0.0543  rd  . 

Consequently,  the  envelope  barrier  does  not  seal  the  "hole"  left  by  the 
natural  barrier. 

For  values  of  p  sufficiently  larger,  5)  does  close.  The  limiting 
value  pQ  has  been  numerically  found  to  be 

p  -  1.062  . 
o 


this  quantity  satisfies,  since 


V  cos 


ii)  The  Envelope  Barrier.  To  compute  an  incoming  trajectory,  we 
need  the  adjoint  vector  v  at  each  point  of  S)  ,  vy  and  can  be 

obtained  from  the  fact  that  v  is  normal  to  ®  .  Then,  the  third  com¬ 
ponent  v  can  be  obtained  from  H,  =  0  for  the  three-dimensional  game, 
r  1 

In  the  cylindrical  system  of  coordinates,  we  have 


The  first  of  the  following  two  relations  comes  from  the  fact  that  v  is 
normal  to  S)  .  Placing  it  in  the  above  equation  and  rearranging, 
becomes  a  perfect  square  and  yields  the  second  relation: 


V  c2-a2 

Vv  =  "  f  ve 

1  v  cos  9 

Vr  "  7Aa:v2C0Sie-  ve  • 


(5.9) 
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It  is  convenient  to  introduce  the  parameter  p 


v  =  p  cos  (y-0) 
r 

Vg  =  rp  sin  (y-0) 

and  relation  (5.9)  yields  y  ,  and  consequently  all  three  adjoints, 
through 

w  cos  (0-y)  =  v  cos  0  .  (5.9a) 

Notice  that  a  consequence  of  this  last  relation  is  that  at  B  ,  where 

v  cos  0  =  w  ,  0  =  X  •  Therefore,  =  V  =0  and  v  =  P  .  We  have 

c?  v  r 

a  verification  of  the  fact  that  v  is  the  same  for  both  barriers  at 
this  point. 

It  has  been  found  that  the  envelope  barrier  is  terminated  by  a  cusp 
of  the  type  already  described,  that  reaches  the  capture  circle  at  D  . 

It  is  an  interesting  problem  to  find  a  way  to  characterize  such  a 
cusp  with  absolute  certitude  when  the  available  data  is  numerical,  and 
therefore  approximate.  In  particular,  it  seems  difficult  to  distinguish 
a  cusp  from  a  "fold"  with  very  small  radius  of  curvature. 

The  solution  lies  in  the  fact  that  together  with  the  trajectories 
defining  our  surface,  we  compute,  with  a  separate  set  of  equations,  the 
normal  vector  v  .  This  allows  us  to  follow  continuously  a  given  side  of 
the  surface,  and  consequently  to  distinguish  between  a  cusp  and  a  finite 
radius  of  curvature,  no  matter  how  small  (see  Fig.  5). 

A  numerical  localization  of  the  cusp  is  possible  with  good  accuracy 
by  computing  a  curve  on  the  surface,  other  than  a  trajectory. 

The  situation  is  now  the  following: 

For  p  <  1  :  We  have  a  smooth  open  barrier  terminated  by  a  cusp. 
Capture  will  occur  from  any  initial  condition,  in  agreement  with 


and  the  angle  X  : 

V  =  P  sin  y 
x 

v  =  p  cos  y 

y 
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FIGURE  5.  Characterization  of  a  Cusp 

the  results  of  Chapter  Four.  The  precise  shape  of  the  optimal 
capture  trajectories  will  be  investigated  in  Chapter  Six. 

For  p  =  1  :  The  natural  barrier  closes  forming  a  crest  on  the  (y,v) 
plane.  But  the  trajectories  of  the  envelope  barrier  never  reach 
that  plane.  Thus,  the  two  symmetric  parts  of  this  barrier  are 
tangent  at  A'  and  separate  toward  lower  v's  . 

For  1  <  p  <  pQ  :  The  envelope  barrier  forms  a  crest  on  part  or 
all  of  its  length.  It  is  still  open  at  the  lower  v  end. 

For  p  >  p0  :  The  two  barriers  form  a  continuous  surface  that  seals 
off  a  region  of  the  state  space.  An  evader  coming  from  outside 
this  capture  region  can  always  escape. 

Figure  8  schematically  depicts  the  situation  for  p  =  1  .  The  en¬ 
velope  barrier  has  been  arbitrarily  interrupted  at  y  =  constant  for 
clarity. 

5.5  The  Envelope  Junction 

i)  Motivation.  The  two  barriers  we  know  so  far  correspond  to  a 
chase  in  which  the  evader  side-steps  in  an  attempt  to  outmaneuver  the 
pursuer.  Curve  J)  terminates  for  small  v's  because  the  pursuer  is 
then  too  maneuverable.  We  expect  that  the  evader  will  take  advantage  of 
his  greater  speed  and  essentially  flee  from  the  pursuer. 
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FIGURE  6.  Envelope  and  Natural  Barriers 


Consider,  in  particular,  a  chase  starting  in  the  symmetry  plane,  at 
low  v  .  It  is  intuitively  clear  that  both  players  will  direct  their 
controls  in  the  direction  of  the  vector  v  .  The  ensuing  motion,  a 
straight  chase  in  the  physical  space,  will  appear  in  the  state  space  as  a 
parabola  in  the  (y,v)  plane,  the  equations  of  which  are  given  by 


*1  = 
dt 


w 


-  v 


d v 
dt 


=  F 


that  integrates  into 


y  - 


If  p  >  1  ,  the  barriers  reach  the  (y,v)  plane.  One  of  these  parabolas 
just  reaches  the  crest  and  provides  escape.  It  should  be  part  of  a  bar¬ 
rier,  since  a  parabola  immediately  under  it  fails  to  reach  the  barrier. 


For  the  worst  case:  p  =  1  ,  y  =  £ 

o 

tive  vq  .  Precisely,  it  must  go  through 


still  corresponds  to  a  posi- 
A'  ,  which  gives 


v 

o 


We  notice  that  such  an  escape  trajectory,  if  it  is  to  be  retained  as 
part  of  the  barrier,  presents  a  corner  where  the  parabola  reaches  the 
crest.  Hence  the  need  for  the  equivalent  of  a  corner  condition  for  bar¬ 
riers.  This  is  provided  by  the  following  theory. 

3  l) 

ii)  A  Corner  Condition.  Let  S  and  S  be  two  semi-permeable 
surfaces  intersecting  at  a  non-zero  angle.  Each  of  them  locally  separates 
the  space  into  two  regions:  Ra  and  Ra  for  Sa  ,  and  similarly  for  Sb  , 

JL  w 

the  subscripts  being  determined  by  the  direction  of  v  (we  purposely 
avoid  specifying  whether  v  points  into  region  one  or  two) .  The  com¬ 
posite  surface  locally  separates  the  space  in  two  regions  R^  and  Rp  . 

Let  us  say  that 


R1  -  Ral 

nnj 

(dihedron  less 

than 

it  ) 

»2 

(dihedron  more 

than 

Jt  ) 
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The  composite  surface  S  is  obtained  by  discarding  the  por- 
.  a  b 

tions  of  S  and  S  lying  in  R2  .  Let  be  the  control  of  the 

player  who  tries  to  go  into  region  1,  and  cp  the  other  player's  control. 
We  make  the  following  assumption: 

Assumption:  On  semi-permeable  trajectories  arriving  at  J  =  Sa  Pi  Sb  , 
condition  (5.1)  uniquely  defines  cp^  . 

Under  this  hypothesis  we  prove  the  following  theorem: 

Theorem:  For  S  to  be  a  barz’ier,  it  is  necessary  that  the  trajectories 
incoming  to  the  junction  do  not  cross  it.  They  must  either  be  tangent  to 
it  or  present  a  corner. 

Proof:  Let  us  assume  the  contrary:  some  paths,  say  in  Sa  ,  actually 

cross  J  .  Two  situations  can  occur  at  J  : 

1)  <p^  ^  9^  •  When  the  state  reaches  J  ,  player  2  will  keep  his 

a  a 

strategy  <p2  .  If  player  1  keeps  his  strategy  <p  ,  by  the  current  hy¬ 
pothesis  he  will  let  the  state  penetrate  Rb  C  R  .  If  he  plays  any 
other  strategy,  by  our  previous  assumption  he  will  let  the  state  penetrate 
R^  C  R2  .  In  every  case  the  state  penetrates  R2  ,  and  S  is  not  a 
barrier. 

2)  <pa  =  cpb  .  Then  when  reaching  J  on  S3  ,  player  1  has  a  con¬ 
trol  that  prevents  crossing  of  Sb  ,  and  consequently  of  J  .  Therefore, 
the  trajectoi’ies  of  Sa  cannot  cross  J  . 

T.iis  ends  the  proof. 

Remark  1:  <pa  =  <pb  i-  not  necessary.  A  pair  of  strategies  (cpa,<p2) 

can  generate  paths  reaching  Sb  tangentially  without  being  equal  to 
b  b. 

(q> , cp2 )  at  J  .  In  that  case,  the  theorem  states  that  the  trajectories 
of  Sa  will  fall  back  into  R^  .  Player  2  is  thus  obliged  to  change  his 
strategy.  He  has  the  choice  between  two  possibilities: 

.  b  b 

1)  Either  switch  to  <p  ;  then  player  1  will  switch  to  cp  ,  and  the 

^  b  *"* 

game  will  follow  0  trajectory  of  S  ,  which  supposedly  leaves  J  ; 

2)  Or  vary  <p  so  as  to  be  always  in  accordance  with  cpa  on  the 
incoming  trajectory  at  J  .  Player  1  must  then  choose  the  corresponding 
control  cpa  ,  and  the  state  will  traverse  J  . 
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Remark  2:  J  is  for  S  a  cusp  of  the  type  discussed  earlier,  so  that 

g 

S  actually  comes  to  an  end  on  J  .  The  same  reasoning  as  in  our  proof 
applies  to  the  junction  of  a  barrier  with  the  non-usable  part  of  C  and 
gives  the  envelope  barrier.  In  that  case,  the  surface  we  join  on  is  not 
semi-permeable,  and  the  evader  has  an  infinity  of  strategies  that  prevent 
crossing  it.  But  we  needed  to  assume  the  unicity  of  (p  on  the  incoming 
trajectories  only. 

iii)  The  "Roof"  for  p  =  1  .  For  p  =  1  ,  the  parabola  we  have 

described  above  is  tangent  to  the  barrier  at  A'  .  It  is  thus  natural  to 

construct  an  envelope  junction  from  A*  on  the  envelope  barrier  S  .  As 

we  do  not  have  the  analytical  expression  of  the  envelope  barrier,  finding 

the  junction,  say  J  ,  as  a  barrier  of  the  game  constrained  to  lie  on 
g  is  not  feasible.  We  choose  a  different  approach: 


Let  f(v,z)  be  the  direction  defined  by  the  controls  verifying 


min  max  (v,f  (z,<p,ty) )  =  (v,f(v,z))  . 
cp  y 

Let  v  be  the  vector  v  on  g  .  The  problem  is  to  find  whether  the 
equations 


(v,f (v»z) )  =  0 
(vb,f(v,z))  =  0 


(5.10) 


3  b 

have,  for  a  given  z  ,  a  solution  v  £  V  •  The  first  equation  says 
that  v  belongs  to  the  local  cone  of  semi-permeable  normals,  and  the 
second  one  that  the  corresponding  direction  is  tangent  to  §  .  We  are 


looking,  in  the  cone  of  semi-permeable  directions,  for  a  direction  tangent 
to  g  ,  other  than  that  of  the  trajectory  of  g  . 


If  such  a  direction  exists,  for  z  in  some  neighborhood  of  A'  ,  we 
can  consider,  on  the  surface  S  ,  the  field  of  directions  f(v  ,z)  ,  and 
integrate  it  as  a  differential  equation.  As  we  are  looking  for  a  curve 
lying  on  the  envelope  barrier,  which  is  known  only  numerically,  carrying 
out  this  program  presents  some  technical  difficulties.  The  ideas  of  the 
numerical  method  used  are  outlined  in  Appendix  B. 

In  our  case,  equations  (5.10)  can  be  made  simple,  introducing  the 
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parameters  p  and  y  as  in  the  previous  sections,  and  a  =  Q-y  ,  the 
angle  between  (\,v  )  and  r  .  (Notice  that  ty*  =  y  .) 

They  become: 

H1  =  -Fa  +  p(w-v  cos  y)  =  0 


2  jj  22  b  2  u 

-F(.v  vvvy  +  p  r  sin  a  sin  a )  +  pv  a[w  cos  (y-y°)  (5.11) 


-  v  cos  y  ]  =  0  . 


To  avoid  difficulties  in  the  case  =  0  ,  we  solve  the  first  equation 
for  vv  and  put  it  in  the  second  one.  And  we  look  for  the  roots  in  y 
of  the  equation: 


_2  b 
F  vv 

v 


2  2  2  2  2  b 

v  cos  y)  -  r  sin  a  -  F  Pr  sin  a  sin  a 


2  2  l)  b 

+  v  p  (w-v  cos  y)[w  cos  (y-y )  -  v  cos  y  ]  =  0  (5.11a) 


where,  we  recall, 

a  =  e-y  ab  =  e-yb  (pb  =  p)  . 

It  can  be  checked  that  Hb  =  0  implies  that  this  equation  admits  the 
b  1 

root  y  =  y  ,  but  we  want  a  different  one.  At  A  ,  y  =  0  is  the 
desired  root.  For  other  points,  this  equation  was  sol.ved  numerically  by 
Newton's  technique. 


We  were  able  actually  to  compute  a  junction  ‘J  and  the  corresponding 
semi-permeable  surface  !R  .  The  main  feature  is  that  ^  meets  the  cusp 
terminating  6  ,  and  not  3)  ,  at  a  point  J  : 

x  =  0.772  10~2  £  y  =  1.37287  £  v  =  0.6438  w  . 

Consequently,  the  "roof"  $  ,  while  it  does  seal  the  hole  between  S  and 
the  symmetry  plane,  still  leaves  a  "hole"  in  the  barrier  between  the  end 
of  S  and  the  trajectory  of  51  arriving  at  J  .  This  situation  is  de¬ 
picted  in  Fig.  7. 
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FIGURE  7 .  The  Roof  K  (  x  >  0  only  is  drawn) 


5.6  Singular  S.P.S. 
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i)  Motivation.  For  1  <  p  <  ,  where  p„  -  1.092  ,  the  envelope 

-  i  -  -  ^  £ 

barrier  forms  a  crest  on  part  of  its  length,  but  the  end  of  this  crest 
has  a  smaller  slope  than  the  parabola  at  that  point.  Consequently,  this 
parabola  cannot  be  part  of  a  barrier,  since  after  reaching  S  it  falls 
back  in  the  escape  region,  violating  our  necessary  condition.  Also,  and 
this  is  closely  related  to  the  previous  fact,  a  parabola  immediately 
"under"  it  still  reaches  the  barrier  and  provides  escape,  showing  that 
the  previous  one  was  not  a  limiting  escape  trajectory. 

Loosely  speaking,  on  that  parabola  the  evader  was  too  strong.  The 
limiting  one,  the  one  that  verifies  our  theorem,  is  the  parabola  that 
reaches  the  crest  tangentially,  and  this  determines  the  point  where  the 
roof  must  attach.  However,  a  new  problem  arises. 

If  from  that  point  we  apply  the  previous  construction,  we  still  find 
a  junction  $  and  a  corresponding  roof  !R  .  But  now,  we  start  from  a 
point  where  &  is  not  tangent  to  the  symmetry  plane.  Consequently,  the 
trajectory  of  !R  reaching  the  crest  does  not  lie  in  that  plane,  but 
reaches  it  at  a  non-zero  angle.  Therefore,  the  envelope  roof  is  now  made 
of  two  symmetric  strips,  leaving  a  hole  between  these  two,  in  addition  to 
the  hole  already  described  in  the  case  p  =  1  . 

In  particular,  we  have  not  found  what  semi-permeable  surface  our 
parabola  is  imbedded  in.  To  solve  this  problem,  we  need  a  slightly  new 
concept. 

ii)  The  Singular  S.P.S,  We  have  seen  that,  at  each  point  of  the 
state  space  there  is  a  cone  of  semi-permeable  directions.  We  claim  that 
the  family  of  trajectories  generated  by  such  a  cone,  by  backward  integra¬ 
tion,  is  in  fact  a  semi-permeable  surface.  We  call  such  a  surface  a 
singular  S.P.S.,  referring  to  the  point  where  all  its  trajectories  meet 
as  its  singular  point. 

That  it  is  a  semi-permeable  surface  can  be  seen  by  the  fact  that 
each  vector  v  at  the  singular  point  is  normal  to  a  tangent  plane  of  the 
cone  at  this  point.  This  is  a  consequence  of  our  geometric  construction 
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of  this  cone  as  the  envelope  of  a  family  of  planes  H  normal  to  the  v's  . 
Then,  the  transformation 

.  .  c)f  . 

5z  =  ^  5z 

transforms  such  a  plane  into  a  tangent  plane  at  an  ordinary  point  of  the 
surface,  and  v  being  governed  by  the  adjoint  equation  will  still  have  a 
constant,  thus  zero,  dot  product  with  5z  . 

This  proof  is  based  upon  our  geometrical  theory  of  the  semi-permeable 
cone.  But  the  same  fact  can  be  seen  in  Isaacs’  analytical  theory,  con¬ 
sidering  his  theorem  on  the  construction  of  semi-permeable  surfaces  ([18], 
theorem  8.3.1,  p.  208).  He  parametrizes  the  initial  curve  with  s  and 
proves  that 

3z . 

Rk  =  £  Vj  ^ 

3  k 

is  a  constant.  If  it  was  zero  at  r  =  0  ,  it  remains  zero  for  every  t  . 
This  can  happen  in  two  different  ways: 

either  v  is  normal  to  the  line  z(s)  (ordinary  semi-permeable 

surface) 

or  bz./bs  =0  Vi  (singular  semi-permeable 

1  surface)  . 

We  are  going  to  employ  this  concept  to  complete  the  roof. 

iii)  Completion  of  the  Roof.  Observe  that  at  the  end  of  the  crest, 
incoming  trajectories  need  not  be  tangent  to  S  .  As  long  as  their  pro¬ 
longation  falls  back  into  the  capture  region,  as  is  the  case  for  the 
parabola  we  retained,  they  do  not  violate  our  necessary  condition. 

From  the  point  where  we  attached  {I  ,  we  can  construct  a  singular 
surface,  limiting  it  to  those  trajectories  that  are  tangent  to  6  ,  and 
thus  belong  to  !R  .  All  the  trajectories  of  this  family  fall  within  the 
conditions  of  our  theorem.  This  surface  provides  a  smooth  extension  to 
the  previously  constructed  roof,  and  completes  it  toward  the  symmetry 
plane. 
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Construction  of  the  roof  was  carried  out  numerically  for  vaz’ious 
values  of  the  parameter.  The  shape  of  the  surface  obtained  depends  on 
this  parameter.  We  found  that  two  limiting  values  p  and  p0  have  to 
be  considered.  Precise  numerical  determination  of  these  values  is  dif¬ 
ficult,  due  to  low  sensitivities.  According  to  our  calculations,  they 
are  in  the  ranges 

1.056  <  p,  <  1.057  <  p  =  1.062  <  1.091  <  p„  <  1.092  . 

1  O  2 

For  1  <  p  <  p^  ,  we  have  the  same  qualitative  situation  as  for 
p  =  1  .  The  envelope  junction  reaches  the  cusp  on  §  .  The  roof  seals 
the  hole  between  the  envelope  barrier  and  the  symmetry  plane,  but  leaves 
two  symmetrical  ones  between  its  last  trajectory  and  the  cusp  on  S  . 

For  p  <  P  <  P„  ,  the  envelope  junction  reaches  the  curve  ®  .  We 
have  a  closed  capture  region  delineated  by  the  natural  barrier,  the  en¬ 
velope  barrier  and  the  composite  roof.  Notice  that  the  smallest  value  of 
the  parameter  for  which  the  capture  region  is  closed  has  been  taken  down 
from  pQ  to  p^  ,  closure,  between  these  two  values,  being  provided  by 
the  roof. 

For  p  <  p  ,  the  crest  has  a  slope  larger  than  that  of  the  para- 
bolas  on  all  of  its  length.  No  roof  occurs.  The  natural  barrier  and  the 
envelope  barrier  together  define  a  closed  capture  region. 

5.7  The  Alain  Singular  Barrier 

i)  Junction  of  Three  Surfaces.  From  the  point  J  where  the  en¬ 
velope  junction  {j  meets  the  cusp  on  &  ,  we  can  generate  another  singu¬ 
lar  surface,  A  ,  that  provides  a  smooth  extension  to  the  roof  toward  the 
"side."  The  question  of  whether  this  semi-permeable  surface,  together 
with  the  rest  of  the  barrier,  still  forms  a  barrier  turns  out  to  be  dif¬ 
ficult  in  two  respects.  The  first  problem  has  to  do  with  what  happens  at 
J  .  It  will  be  discussed  here.  The  second  one  has  to  do  with  the  inter¬ 
section  of  A  and  $  .  It  will  be  mentioned  at  the  end  of  this  sub¬ 
section,  discussed  in  Section  5.8,  and  again  at  the  end  of  Chapter  Six. 

At  J  ,  the  trajectories  of  A  seem  to  violate  the  necessary  condi- 
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tion.  Our  theorem,  however,  was  for  the  junction  of  two  ordinary  sur¬ 
faces,  and  did  not  exclude  the  possibility  of  a  third,  singular  one. 

In  this  instance,  it  turns  out  that  the  direction  fa  of  the  roof 

g 

and  the  vector  v  of  h  verify 
<vs,fa>  >  o  . 

It  means  that  for  this  pair  of  surfaces,  it  is  the  pursuer  who  has 
the  advantage  of  the  larger  region  (region  2  of  the  theorem). 

Then,  upon  arrival  at  J  on  a  trajectory  of  ^  ,  the  sequence  of 
decisions  is  as  follows: 

1)  Since  the  trajectory  extends  into  the  escape  region,  the  pursuer 
switches  to  the  roof  strategy. 

2)  If  the  evader  does  not  switch,  he  will  let  the  state  drift  below 
the  roof.  If  he  switches  to  the  roof  strategy,  he  places  the 
state  on  a  trajectory  that  goes  back  into  the  capture  region. 

Thus  he  must  choose  the  envelope  barrier  or  the  envelope  junction 
strategy. 

3)  Thus  the  pursuer  is  obliged  to  switch  again  to  counter  the  evader. 

If  the  pursuer  had  switched  to  the  envelope  barrier  strategy  to  start 
with,  without  the  evader  switching  first,  he  would  have  let  the  state  go 
above  the  roof  in  the  escape  region.  On  the  other  hand,  the  evader  could 
not  directly  switch  to  the  envelope  barrier  strategy,  because  then  the 
pursuer  would  not  have  switched  and  the  state  would  have  gone  under  the 
singular  barrier  in  the  capture  region. 

The  effect  of  the  various  choices  can  be  shown  in  a  diagram.  We 
have  plotted  horizontally  the  three  choices  of  the  pursuer,  by  the  name 
of  the  surface  he  plays  according  to,  and  vertically  the  choices  of  the 
evader.  In  each  box,  we  have  written  P  or  E  according  to  whether  P 
or  E  wins  with  that  combination.  N  stands  for  neutral.  The  arrows 
indicate  the  sequence  we  have  described. 

Several  remarks  must  be  made  about  this  description. 


101 


First,  notice  that  we  are  obliged  to  assume  that  the  players  know 
each  other's  control.  This  is  not  a  very  serious  problem.  One  can,  for 
instance,  say  that  an  infinitesimal  loss  is  acceptable  to  them,  and  that 
the  motion  of  the  state  during  an  infinitesimal  time  gives  them  the 
necessary  information. 

Observe  also  that  the  pursuer's  last  move  could  have  been  replaced 
by  his  going  back  to  the  singular  barrier  strategy.  But  then  the  evader 
would  switch  again,  and  we  could  have  an  infinite  cycling,  all  supposed 
to  be  instantaneous!  This  is  because  what  we  have  is  a  matrix  game  with 
no  saddle  point.  We  deliberately  exclude  the  consideration  of  mixed 
strategies,  which  would  anyway  be  of  little  help  in  a  qualitative  game. 

To  solve  this  problem,  we  assume  that  both  players  prefer  the  neutral 
outcome  to  the  risk  of  letting-  the  opponent  take  the  better.  Then,  the 
natural  sequence  of  decisions  is  the  one  we  proposed. 

The  second,  and  much  more  difficult,  problem  arises  at  this  point. 
The  envelope  barrier  trajectories  can  be  considered  as  falling  back  into 
the  capture  region  defined  by  the  singular  barrier.  In  fact,  the  P  and 
the  N  in  the  last  row  of  our  matrix  are  not  firmly  established.  This 
will  be  discussed  in  the  next  section. 

ii)  Shape  of  S-  .  We  must  investigate  the  qualitative  shape  of 


102 


A  ,  and  see  whether  it  seals  the  hole. 

The  first  fact  is  that,  whereas  the  trajectories  close  to  the  roof 
look  like  the  roof  and  come  from  the  capture  cylinder,  when  we  go  to  dif¬ 
ferent  enough  directions  (larger  y’s  ),  we  find  trajectories  that  do  not 
come  from  the  capture  set.  Moreover,  we  find  that  this  surface,  too,  is 
terminated  by  a  cusp,  where  the  backward  computation  of  those  trajectories 
must  be  stopped.  This  cusp  does  not  touch  C  . 

The  situation  at  this  point  is  depicted  by  Fig.  8,  where  we  purposely 
avoided  specifying  what  happens  at  the  intersection  of  A  and  S  ■ 

5.8  Another  Envelope  Barrier,  Discussion 

i)  The  Barrier  S'  .  If  A  is  actually  part  of  the  barrier,  it  is 
easy  to  find  yet  another  smooth  extension  to  it.  Let  B'  be  the  point 
where  a  trajectory  of  A  is  tangent  to  C  .  Note  that  according  to  what 
we  said  in  Section  5.4,  B*  has  to  lie  in  the  region  <  0  since  at 
that  point  there  is  a  semi-permeable  direction  tangent  to  C  . 

This  direction  can  be  imbedded  in  a  family,  as  we  argued  when  we 
constructed  the  curve  5)  .  We  must  just  take  the  other  sign  for  <p  , 
giving  a  decreasing  v  .  We  can  in  this  way  compute  in  C  a  curve  £>' 
given  by  equation  (5.8)  but  with  the  opposite  sign: 

de 

dv 

and  joining  along  this  curve,  a  new  envelope  barrier  §'  can  be  con¬ 
structed.  The  formulas  for  the  v's  are  the  same,  except  for  a  minus 
sign  in  v  • 

In  the  vicinity  of  B'  ,  §'  blends  smoothly  into  A  ,  since  the 

v's  are  continuous  across  B1  .  It  has  been  found,  however,  that  start¬ 
ing  at  the  point  where  the  trajectory  through  B'  touches  the  cusp, 

6'  intersects  A  at  a  non-zero  angle,  cutting  off  the  cusp.  If  we 
delete  both  surfaces  beyond  this  intersection,  we  have  a  simple  dispersal 
line  of  the  game  of  kind. 
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FIGURE  8.  The  Singular  Barrier 


g'  also  intersects  g  ,  as  ©'  intersects  ®  .  Figure  9  is  an 
attempt  to  describe  this  set  of  surfaces. 

ii)  Discussion  .  We  have  a  system  of  semi-permeable  surfaces  ap¬ 
parently  separating  a  closed  capture  region  from  an  escape  region.  But 
the  situation  is  more  complicated  than  that. 

As  shown  by  Fig,  9,  there  is  a  common  point  to  g'  ,  A  and  the 
natural  barrier.  From  that  point  to  J  ,  there  must  exist  a  line  of  in¬ 
tersection  of  A  and  the  composite  natural  envelope  barrier.  The  tra¬ 
jectories  of  &  penetrate  A  along  this  intersection,  and  the  necessary 
condition  of  Section  5.5  is  violated.  Therefore,  the  set  of  surfaces  we 
have  described  does  not  constitute  a  barrier. 

The  problem  cannot  be  solved  by  discarding  the  part  of  g  "above" 

S  ,  as  the  existence  of  %  and  A  itself  is  based  upon  the  existence 
of  g  .  A  conjecture  will  be  presented  later  as  to  how  the  barrier  may 
look.  Using  the  game  of  degree,  a  part  of  A  would  be  discarded,  g' 
would  be  kept  complete  as  a  barrier,  and  not  truncated  where  it  inter¬ 
sects  A  . 

Then  we  still  have  trajectories  of  g  crossing  the  intersection 
with  g'  .  However,  this  is  not  a  contradiction  for  the  following 
reason:  along  this  intersection,  the  "escape  region"  is  the  region  out¬ 

side  of  both  g  and  g'  .  When  seen  as  such,  the  intersection  has  only 
trajectories  leaving  it.  The  part  of  g  inside  g'  merely  defines  the 
region  for  which  g1  is  a  barrier.  Inside  of  g  ,  g'  does  not  exist. 

This  conjecture  is  depicted  by  Fig.  10. 

5.9  Conclusion 

Our  investigation  of  the  game  of  kind  can  be  summarized  as  follows: 

p  <  1  The  natural  barrier  and  the  envelope  barrier  together  form 

an  open  barrier  terminated  by  a  cusp.  Capture  occurs  from 
any  initial  condition. 

1  <  p  <  Pl  We  have  not  been  able  to  display  a  barrier  sealing  off  a 

region  of  the  state  space.  We  conjecture  that  an  open  bar- 


rier  exists,  made  of  six  intersecting  semi-permeable  sur¬ 
faces,  plus  their  image  in  the  symmetry  plane.  Following 
these  surfaces  could  involve  a  six-stage  chase. 

We  have  a  closed  capture  region;  an  evader  starring  far 
enough  away  will  always  escape.  The  configuration  of  the 
barrier  depends  on  the  relative  values  of  p  and  p^  : 

p  <  P2  :  The  barrier  involves  the  natural  barrier,  the  en¬ 
velope  barrier,  the  envelope  roof  and  the  singular  roof. 

p2  <  p  J  The  roof  no  longer  exists.  The  closed  barrier  is 
made  of  the  two  surfaces  found  by  Isaacs. 


FIGURE  9.  All  the  Barriers 


6.  THE  GAME  OF  DEGREE 


The  previous  chapter  was  concerned  with  the  qualitative  problem: 
capture  or  escape.  It  yielded  surfaces  of  discontinuity  of  the  qualita¬ 
tive  problem  or  "game  of  degree"  we  want  to  investigate  now.  What  are 
the  optimal  strategies  and  the  corresponding  capture  time  and  trajec¬ 
tories? 

6.1  The  Hamilton-Jacobi  Equation 

i)  The  Problem.  Let  the  capture  region  as  defined  by  the  previous 
chapter  be  £' .  When  the  barrier  is  open,  is  the  whole  state  space, 

deprived  of  the  capture  set. 

■V  ^ 

We  are  looking  for  a  pair  of  strategies  (p  (z),  \Jf  (z)  defined  in  the 
capture  region,  and  such  that 

1)  cp*(z)  €  *  y*(z)  €  Y  Vz£  e* 

2)  The  equation  of  motion 
z  =  f(z,cp*  (z),ij/*(z)) 

has  a  solution,  not  necessarily  unique,  lying  in  g'  for 
every  z(0)€g' 

3)  These  solutions  transfer  z(0)  to  the  terminal  manifold  Q 

in  a  finite  time,  and  yield  a  uniquely  determined  payoff 
■X*  ■K’ 

J(z,cp  ,ijr  )  =  V(z),  (in  our  case,  J  is  simply  the  time  of 
capture),  verifying 

min  max  J(z,<p,i|0  =  max  min  J(z,(p,f)  =  V(z) 

<P(->  *(•)  *(•)  (O 

The  main  tool  used  in  the  solution  of  this  problem  is  a  generalized 
version  of  the  Hamilton-Jacobi  equation.  It  was  first  derived  by  Isaacs 
who  called  it  the  "main  equation".  His  derivation  can  be  found  in  [18). 
Berkovitz  [l]  gave  a  more  rigorous  derivation,  using  a  variational  tech¬ 
nique.  Several  other  authors  gave  proofs  of  varying  generality.  A 
recent  one  we  already  referred  to  can  be  found  in  Blaquiere  Gerard ,  and 
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Leitmann  [3]  uses  Leitmann's  geometrical  theory  of  optimal  process,  and 
is  similar  to  Isaacs'  second  proof. 


We  shall  give  a  simple  derivation,  valid  only  under  too  strong 
assumptions.  It  follows  Caratheodory 's  method,  as  did  Chattopadha y  [10] 
for  a  slightly  different  case.  Our  aim  is  to  point  out  that  the  exis¬ 
tence  of  a  saddle  point  in  the  Hamiltonian  is  sufficient  to  insure  that 
the  game  itself  has  a  saddle  point. 

We  directly  arrive  at  a  sufficient  condition,  reached  by  Isaac 
through  his  verification  theorem,  and  by  Berkovitz  through  a  generaliza¬ 
tion  of  Hilbert's  invariant.  Notice  that  Berkovitz'  technique  being 
variational  yields  the  Euler -Lagrange  equations.  Hence  the  need  of 
Hilbert's  invariant  to  show  that  they  are  the  characteristics  of  a 
partial  differential  equation.  In  every  case  sufficiency  arises  from 
the  consideration  of  a  field  of  extremals.  (See  [5]). 

ii)  Derivation.  We  consider  the  more  general  case  of  a  non¬ 
stationary,  integral  payoff  game,  defined  by 

z  =  f  (z,cp,t,t) 

f** 

J  =  K(z,t)  z€  +  J  L(z,Cp,V,t)  dt  .  (6.1) 

*o 

where  tf  is  the  first  instant  such  that  z(t)€(r,  the  terminal  mani¬ 
fold  (possibly  time  varying). 

We  define  the  Hamiltonian  function 

H(z,  A,<P,ty,t)  =  L(z,cp,t,t)  +  <A,f(z,<p,V,t)> 

Assume  that 

min  max  H(z,A,<p,>|r,t)  =  max  minH(z,A,cp,\Jr,t)  =  H  (z,A,t) 

<p  V  t  <P 

and  that  this  extremum  is  attained  for  a  uniquely  defined  pair  of  con¬ 
trols,  except,  possibly,  on  some  singular  manifolds.  Let  these  controls 
be 


110 


cp*  =  cp*  (z ,  X,  t ) 


(z ,  A,  t ) 


verifying  conditions  1)  and  2)  above. 

Consider  the  Hamilton-Jacobi  equation 

+  jf(2,  ^  ,  t)  =  °  (, 

Theorem:  Under  the  above  assumptions,  if  the  Hamilton-Jacobi  equation 
(6.2)  has  a  continuously  differentiable  solution  V(z,t)  the 
restriction  of  which  to  6  verifies 


V(z,t) |  =  K(z,t) 

€ 

and  if  the  restrictions  to  a  suitable  interval  (t  ,  t  )  of  the 

o  f 

strategies 


transfer  (z  ,t  )  to  £  at  t_,  then  these  strategies  are  opti- 
o  o  I 

mal  in  the  sense  of  section  6.1,  and  the  corresponding  payoff  is 

V(z  ,t  ). 
o  o 

Proof:  To  prove  this  result,  we  first  establish  Caratheodory 's  lemma: 

Lemma :  Let  the  scalar  function  N(z,  <p,ip,t)  have,  for  every  z  and  t 
a  unique  saddle  point  equal  to  zero,  at  (p*(z,t)  and  ty*(z,t) 

N(z,<p,i|r*(z,t),t)  >  0  ¥  cp  £<i>,  rp^cp*(z,t) 

N(z,<p'x'(z,t)  ,  ty*  (z,t),t)  =  0 

N(z ,  <p*(z ,  t)  ,i]r,  t)  <0  V  £  f*(z,t) 

and  assume  that  (cp^.ilc*)  transfers  the  state  (z^O  to  <? 
at  tp,  then  the  game  with  payoff 

I 
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I  = 


/tx 


N(z,cp,\Jf,t)  dt  z(t][)  £  C 


admits  (cp  )  as  optimal  strategies  for  the  initial  con¬ 
ditions  (zo>to),  and  the  optimal  payoff  is  1=0. 


Proof  of  the  Lemma.  It  is  a  simple  matter  to  check  that,  for  any  t^ , 


1(2 


*  A 

,.<p  .t,t  >  =  j 


N(z,<p*,\{f,t)  dt  <  0 


I(zQ,cp  *^*,tQ) 


N(z,cp*,ty*,t)  dt  =  0 


I(zo,cp,r,to) 


A 

=  J  N(z,(p,ty*,t)  dt  >  0 
t 


hence  I  verifies  (6.2)  and  the  lemma  is  proved.  Then, 
introduce 


„  T  Sv  /av  a 
N  =  L  +  5t  +  W  f/ 


where  V(z,t)  verifies  (6.2).  Clearly,  by  definition  of  H  , 
N  verifies  the  assumptions  of  the  lemma. 

dV 

Next,  observe  that  along  a  trajectory,  N  =  L  +  —  ,  so  that 

dV  dt 

—  being  continuous 
dt 


dt  =  V(zfItf)  -  V (z 


,v/ 


L  dt 


and  since  V(z;f,tf)  =  K(z,t),  we  have 


I  =  J  -  v<vto) 
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As  V  is  a  function  of  time  and  state  only,  and  not  of  cp  and 

ty,  finding  the  minimax  of  I  for  a  given  (z  ,t  )  is  equivalent 

o  o 

to  finding  that  of  J.  We  apply  Carathe'odory 's  lemma  to  I,  and 
recall  that  its  optimal  value  is  zero.  We  immediately  obtain: 

J  has  a  saddle  point 

The  optimal  strategies  are  cp*  and  ty* 

The  optimal  payoff  is  J  =  V(z  ,t  ) 

o  o 

which  proves  the  theorem. 


Remark:  In  our  case,  f,  and  consequently  H,  is  separated: 


H  =  He(z,i!0  -  Hp(z,cp) 

so  that  we  are  assured  that  it  has  a  saddle  point. 

It  is  well  known  that  equation  (6.2)  can  be  solved  using  the  method 
of  characteristics,  which  yields  the  Euler-Lagrange,  or  canonical,  equa¬ 
tions 


dH* 

z 


|5|  *  ,  ,* 

dAJ<p=cp  ,ijr=4 


.  Sh* 


dn 


<P=cp*,\}r=f* 


(6.3) 


where,  as  we  shall  do  from  now  on,  we  represent  the  gradient  of  V  by 
the  symbol  7\.  We  call  its  components  the  adjoint  variables. 

It  is  interesting  to  notice  that  the  semipermeable  condition  (5.1) 

can  be  regarded  as  the  limit  of  (6.2),  where  <)V/dt  =  0,  when  the 

gradient  of  V  is  infinite.  Then,  the  term  in  <^Vz,f)>  is  predominant, 

and  by  rescaling  to  a  finite  v  such  that  7\  =  v/v  .  and  letting  v 

o  o 

go  to  zero,  the  term  in  L  disappears. 

This  corresponds  to  nonregular  points  of  the  linear  theory,  and  to 
the  classical  abnormal  problem  of  calculus  of  variations. 
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6.2  The  Primaries 


i)  A  Compact  Form.  We  use  representation  (4.2)  (see  [4  ]).  The 


Hamiltonian  is: 


H  =  1  -  +  wTv  •  a  +  F7\ 

r  r  v 


(6.4) 


the  saddle  point  is  obtained  for 

t  t 

*  _  r  /s  _  v 
a  ~  p  P  0 


where  p  =  |  a  =  [7?J  •  This  yields 
H*  =  l-Fo  +  pw  -  X  *  v* 


The  adjoint  equations  are 


7?  =  0  7?  =  constant 

r  r 


A  =  Tv  A  =  Tv  -  A  i 

v  r  v  v  r 

o 


where  \  is  the  adjoint  vector  A  at  time  of  capture,  and  t  the 

vo  V 

time  to  go  t  =  t^  -t . 

— >2  2 

Now,  the  capture  set  (2:  r  =  i  is  a  surface  of  constant  payoff 

T  =  0.  Therefore  the  gradient  of  t  is  normal  to  C,  which  gives 


(6.5) 


A  =  H  F  A  =  0 
r  ^  o  v 


where  p  is  given  by  H  =  0.  At.  T=  0,  let 


(6.6) 


7?  =  pv  cos  P 

r  o  o 


so  that  H  =  0  gives 


v  cos  0  -  w 
o 


which  is  consistent  with  our  claim  that  along  the  B.U.P.  the  adjoint 
vector  is  infinite. 

The  consequence  of  equations  (6.5)  and  (6.6)  is  that  A  and 
->  ->  r 
A  have  a  constant  direction,  A  being  parallel,  and  opposite,  to 
-»v  ^ 

A^.  This  together  with  (6.4)  gives  the  first  important  result  on  the 
game  of  degree : 

In  an  optimal  chase,  the  evader  runs  in  a  straight  line  in  the 
physical  space,  and  the  pursuer  keeps  his  acceleration  parallel  to  the 
evader's  velocity,  describing  a  parabola. 

We  integrate  the  equations  of  motion  with  the  calculated  optimal 
controls.  (The  subscript  zero  stands  everywhere  for  the  time  r  =  0, 


2 

— >  — >  — >  /\  X 

r  =  ro  +  VoT  "  W£T  "  FP  T 

taking  into  account  that  r^  =  Z$,  we  obtain 

-  WT  -  F  i.)  & 

we  can  eliminate  v*  in  terms  of  v*,  we  find 

o 

?  -  n  =  Q(t)$  Q(t)  =  ^  Ft“  -  wt  +  Z 

we  recognize  equation  (4.7a),  thus  identifying  the  time  to  go  along 
those  trajectories,  the  primaries,  with  the  estimating  function  of  the 
previous  theory. 

ii)  3-D  Representation.  To  investigate  the  shape  of  our  trajec¬ 
tories,  it  is  convenient  to  come  back  to  the  three  dimensional  representa¬ 
tion. 
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As  our  Lagrangian  L  is  constant,  maximizing  H  or  ^  is  the 
same  problem,  and  the  Euler- Lagrange  equations  (6.3)  are  the  same  as 
in  the  game  of  kind. 

Thus  our  trajectories  are  still  solutions  of  (5.4).  The  only  dif¬ 
ferences  are  that  v  must  be  replaced  by  A,  and  that  p,  still  con¬ 
stant  along  a  trajectory,  is  no  longer  arbitrary  but  defined  by 

1 

p  _  - - -  . 

s  cos  p  -  w 

The  detailed  treatment  of  these  equations  is  given  in  Appendix  A.  It  is 
convenient,  to  express  their  solution,  to  introduce  the  parameters 

|  =  s  sin  £ 

„  1 
n  =  s  cos  p  p  =  - 

I]-W 

and  the  equations  of  the  trajectories  and  of  the  ad joints  are: 


x  =  —  Q(x) 

A  =  p  ! 

V 

X  V 

y  =  ■  Q(x)  +  vt 

*  = 

y  v 

(6.7) 

2  2  ^ 
v  =  (§  +  (tj-Ft)  ) 

V  V 

Notice  that  as  expected 

2  2  2 
x  +  (y-vx)  =  Q(x) 

Notice  also  that  equations  (5.5)  appear  as  a  special  case  of  these  with 
4  =  (y/s^-w2  and  *1  =  w. 

For  p  >  1,  all  the  primaries  meet  on  the  line  Q  :  x  =  0,  y  =  vXj^ 
where  Q^)  =  0.  In  the  region  where  the  natural  barrier  exists,  this 
line  is  its  crest.  From  each  point  of  it  there  is  an  infinity  of  opti¬ 
mal  trajectories  yielding  the  same  capture  time.  This  line  is  similar 
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to  the  point  A  of  the  Homicidal  Chauffeur  game.  (See  [18],  [17]).  But 
for  v  < \/w2-Fi,  that  is  beyond  the  point  A'  toward  the  lower  v’s, 
this  line  is  under  the  roof,  inside  the  capture  region.  There,  it 
represents  a  line  of  focal  points.  It  is  known  that  beyond  such  a  point 
the  trajectories  are  no  longer  optimal  (see  [7 J ) .  We  shall,  later  on, 
propose  a  solution  to  this  problem. 

We  must  investigate  whether  the  fit. it  e.f  nr  tmarles  presents  any 
other  singularity,  in  particular  in  tae  case  p  <  1  where  d  does  not 
exist.  We  calculate  the  functional  determinant: 

a  D  ■  y » v.  > 

"  D(t,£,.T]) 


This  determinant  is 


v[^-» +  f  T  q(t)] 

-  ~|Xj  FQ(t)  +  w(t)-Ft)"]  +  V 

_f  2LI2 

V 

L  v  J 

Lv  J 

<"-r>2  QCt) 

1  (t  -  ifT  «T>) 

1 

V 

V 

\  V  ' 

-  (t)-Ft)  Q(t) 

i  (Ft-w)  (t]-Ft)  +  V 

n-Fr 

V 

v 


We  first  look  at  the  case  t)-Ft  =0,  v  =  which  gives 


T]-W 


0 


A 


0 


T 


1 


Q(r) 

V 


(n-w) 


o 


Q(T) 

t 


0 


For  i]  £  Ft,  we  add  some  linear  combinations  (involving  the  coefficient 
l/n-Fx)  of  some  lines  to  others,  yielding  the  same  value: 
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—  (F-r-w)  (n-F't)v 


0 


The  primaries-  correspond  to  t]  >  w,  thus  this  determinant  is  zero  only 
for  Q(t)  =  0.  For  p  <  1,  it  is  never  zero,  the  field  has  no  sin¬ 
gularity. 


The  second  question  to  ask  is  whether  this  field  fills  the  whole 
space.  It  was  seen  in  Section  (4.4)  that  for  every  (x,y,v)  there 
corresponds  a  t,  and  onee  t  is  known,  it  is  straightforward  to 
deduce  from  (6.7)  that 


I  = 


vx 

Q(t) 


+  Ft 


and  t  being  a  root  of 

2  2  2 
X  +  (y-vT)  =  Q(t) 

2  2  2  2 
we  see  that  x  /Q(t)  is  greater  than  one,  and  consequently  that  v  -§ 

is  always  positive. 

6.3  The  State  Constraint 

i)  The  Concept .  Having  identified  t  along  the  primaries  with  the 
estimating  function,  it  is  natural  to  check  whether  the  corresponding 
trajectories  penetrate  the  capture  set,  as  suggested  by  the  behavior 
of  the  barrier. 

2 

Calculating  r  we  find 

r2-£2  =  tP(t)  =  Tpj-  F2t3  -  F(n-w)  r2  +  /(H-w)*  +  |2-Ff)  t+  21  (t]-w)J 
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2  2 

as  expected,  r  =  Z  for  t  =  0.  But  the  polynomial  P(t)  can  have 
a  positive  root.  Actually,  since  P(0)  is  positive,  it  has  two  such 
roots  or  none.  The  limiting  case  corresponds  to  a  trajectory  tangent 
to  C. 

A  whole  portion  of  the  field  must  be  discarded  because  of  this  fact. 
It  would  correspond  to  trajectories  penetrating  (2  and  leaving  it  again 
before  capture.  This  is  sketched  in  Fig.  11,  in  which  the  shaded  region 
is  left  unaccounted  for  by  the  present  construction- 


FIGURE  11.  The  Primaries 


This  problem  was  discovered  by  J.  V.  Breakwell  and  Boardman;  see 
W-  The  solution  is  their  concept  of  "safe  contact,"  that  appears  in 
other  games  as  well.  (See  [63  and  [20]). 
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Let  us  consider  the  general  game  (6.1).  We  have  to  discard  trajec¬ 
tories  because  they  penetrate  (?  at  a  nonzero  x.  If  this  happens  in 
the  usable  part  of  the  capture  circle  it  raises  no  problem  since  in  that 
region  we  know  another  strategy  that  forces  immediate  capture  anyway. 

Let  us  consider  the  case  where  this  happens  in  the  nonusable  part  of 
the  capture  set.  Then,  the  region  where  we  have  no  trajectory  is  adja¬ 
cent  to  this  nonusable  part  of  £ ,  where  E  can  prevent  immediate 
capture. 

In  that  part  of  £,  E  can  in  particular  maintain  the  state  at  the 
surface  of  the  capture  set,  without  penetrating  it.  One  can  in  principle 
chose  a  coordinate  system  such  that  (?  be  the  set  z  <  (?,  and  the 
"safe  contact  condition"  is: 

z  =  f  (z,\Jr,i}f)  =  0  (6.8) 

n  n 

(We  restrict  ourselves  to  a  first  order  constraint.  See  [8].)  and  this 
can  be  viewed  as  a  relation  that  \Jr  has  to  satisfy.  We  assume  that 
the  optimal  paths  include  a  leg  of  this  type,  leaving  C  the  first 
time  they  meet  a  trajectory  of  the  n-dimensional  game  doing  so.  We 
check  a  posteriori  that  this  allows  us  to  construct  a  field  of  extremals 
filling  the  previously  empty  region,  and  to  which  the  verification  theo¬ 
rem  applies. 

Consider  the  reduced  game  in  the  (n-1)  first  state  variables  on 
the  surface  of  (?  .  We  know  its  sensitivity  vector  at  the  point  where 
the  trajectories  already  known  leave  C:  since  V(z)  has  to  be 
uniquely  defined,  it  is  the  projection  of  the  n-dimensional  sensitivity 
vector  at  the  same  point.  (See  the  "jump  condition"  in  [3]  and  [9]). 
Therefore,  we  can  integrate  constrained  trajectories  backward  from  these 
points,  together  with  their  ad joints. 

Then,  at  each  point  of  a  constrained  path,  we  can  compute  a  in¬ 
coming  extremal  of  the  unconstrained  game,  recovering  the  missing  ad¬ 
joint  by  the  main  equation  H  =  0.  If  this  construction  actually  yields 
a  field  of  trajectories  filling  the  void  region,  we  have  reached  our 
objective.  According  to  Issues’  terminology,  the  extremals  of  the  n- 
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dimensional  game  constructed  in  this  way  will  be  called  the  "tributaries" 
of  the  constrained  trajectories. 

Notice  the  obvious  similarity  between  the  construction  and  the  one 
that  led  to  the  envelope  barrier.  The  curve  2)  now  appears  as  a  natural 
limit  of  the  constrained  game,  and  the  envelope  barrier  itself  as  the 
corresponding  family  of  tributaries.  About  this  construction,  we  prove 
the  following  simple  result. 

Definition.  The  state  constraint  (6.8)  is  said  to  be  singular  if  the 
equation 

f  (z,ty*(z,A),  ty*(z,A)  =  0  (6.8a) 

cannot  be  solved  for  Afl  in  terms  of  the  (n-1)  first  A^'s,  and 
z.  For  a  further  discussion  of  the  terminology  and  of  the  relation 
with  known  results  of  calculus  of  variations,  see  Appendix  D. 

Theorem.  If  the  safe  contact  condition  is  not  singular,  the  state  con¬ 
straint  is  reached  (and  left)  tangentially. 

A  A  A 

Proof.  Let  z  =  (z_  , . . . ,  z  .),  A  =  (A,  ...  A  ,),  and  y  denotea 
— — —  jl  n-i  i  n-i 

control  ijf  satisfying  the  safe  contact  condition.  The  dynamics 
of  the  reduced  game  are  ' 

z  =  ?(z,  <p>?) 

Q  A  A 

Its  optimal  strategies  are  given  by  the  functions  <p  (z,A)  .and 
\j/°(z,A)  verifying 


1 
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and  moreover,  f-his  quantity  is  H°  =  0. 
th 

The  n  component  of  A  in.  the  n-dimensional  game,  at  any 
point  of  the  constrained  game  is  given  by  H*  =  0,  where 


H*  =  min  max  |L(z,(p,\|0  +  I  W*  ,9  =  L(z,  (p*,ty*) 


n 

+  ^  Akfk(z,cp^,^)  (6.9a) 

1 

Under  our  hypothesis,  we  have  a  simple  way  to  find  a  solution 

of  this  equation.  Let  A  satisfy 

n 

fn(z,<p*(z,A),  y*(z,  A)  =  0 

For  this  A,  fk(z,<p*,ty*)  =  }^(z,cp*,ty*)  for  every  k  <  n. 

Consequently,  (6.9)  and  (6.9a)  are  identical,,  and  yield  identical 

implicit  functions.  Once  z  and  A  are  known: 

n  n 

cp°  (z,  A)  =  cp*  (z,  A) 

$°(z,A)  =  \Jr*  (z ,  A) 

and  therefore 

H*  =  H°  +  Anfn(z,cp*,^)  =  0 

which  proves  that  this  A  is  the  solution  sought.  Then,  the 
equality  of  the  optimal  controls  on  both  arcs  proves  the  theorem. 

The  proof  can  easily  be  generalized  to  a  multivariable  state  con¬ 
straint.  It  does  not  specify  whether  the  point  considered  corresponds 
to  an  incoming  or  outgoing  trajectory  of  the  n-dimensional  game.  It 
therefore  applies  to  both. 

ii)  The  I.R.G.: Locus  of  Tangency  Points  (See  [4]).  In  our  case, 
there  is  no  such  thing  as  singular  controls,  and  the  theorem  applies. 
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We  look,  therefore,  for  the  locus  of  the  points  where  a  primary  is  tan¬ 
gent  to  £.  This  will  be  given  by  a  simultaneous  solution  of. 

P(T)  =  ~  fV  -  F(t]-w)t2  +  |( rj-w)2  +  |2  -  Fi j  T  +  2i>(T|-w)  =:  0 

P'(t)  =  ~  F2t2  +  2F(T)-w)T  +  (i}-w)2  +  |2  -  Ff  =  0 

The  algebraic  condition  for  this  two  polynomials  to  have  a  common  root 
is  that  their  resolvant  be  zero.  This  gives,  after  some  calculations: 

A  =  fY  (i2  +  £  -  |  Fi)  +  8F.e£2j  "  -J  F V(£2-3|2  +  4F£)  =  0 

2 

where  £  =  j}-w.  This  equation  is  of  second  degree  in  £  ,  and  could 
thus  be  solved  for  £  in  terms  of  Using,  then,  the  suitable  root 

of  P' (t)  =  0  would  give  the  desired  locus. 

But  this  technique  gives  very  complicated  formulas  we  prefer  to  use 

a  different  one:  Take  the  common  root  t  as  the  independent  parameter. 

c 

The  simple  operation 

P(t)  -  TP' (t)  =  0 
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which  are  the  coordinates  of  B.  The  curve  2)  is  the  constrained  trajec¬ 
tory  corresponding  to  Tc  =  0. 

The  locus  of  tangency  points  lying  between  2)  and  the  B.U.P.  ,  who  are 
tangent  at  B,  is  therefore  tangent  to  the  B.U.P.  It  turns  out  to  be 
extremely  close  to  2)  and  numerically  very  difficult  to  separate  from 
it  on  part  of  its  length. 

It  is  clear  that  at  Tc  =  0,  formulas  (6.10)  give 

<LU  =  o 

cU 

If  we  notice  that  the  B.U.P.  2  is  given  by  ^  =  constant,  this  shows 
that  the  locus  of  capture  points  of  the  tangent  trajectories  is  also 
tangent  to  2  at  B. 

For  p  >  1,  let  t  be  such  that  QCt^)  =  0.  Clearly,  the  tangency 
point  corresponding  to  t  =  must  be  at  the  point  of  intersection 

of  the  line  Q  with  <?.  For  p  =  1  the  locus  of  tangency  points  is  tan¬ 
gent  to  the  symmetry  plane,  and  it  intersects  it  for  p  >  1.  However, 
constrained  trajectories  arriving  at  a  point  of  this  locus  with  t  >  t.. 

v  A 

must  be  discarded  because  the  corresponding  primary  would  go  through  Q 
before  capture  occurs,  and  thus  would  not  be  optimal. 

iii)  The  I.K.G.:  Constrained  Trajectories.  We  already  know  that  the 
safe  contact  condition  gives  i}f  independently  of  <p  as 

A  V 

cos  0|r  -  0)  =  —  cos  0 

T  w 

/s  1  r 2  2  2  " 

sin  (ty  -  0)  =  —  w  -v  cos  0 

nnd  that  the  corresponding  dynamics  are  equations  (5.7).  The  Hamiltonian 
is 

H  =  —  |^/w“-v~  cos“  0  +  v  sin  0^  -  sin  cp  -  7^  cos  <pj  +  1 


12-1 


The  optimal  control  is  thus 


o  \ 

sin  cp  =  — 

VO 


cos  c p  =  — —  0  = 


A 

V 

0 


7\ 

1-4  +  A2 
2  v 
V 


which  is  consistent  with  our  claim  that  cp  =  cp  at  every  point.  The 
corresponding  Hamiltonian  is 


^o  \  (  r 2  2  2 

H  =  ~  Iv  sin  0  +lw  -v  cos  6  1  -  Fo  +1=0 


and  the  path  and  adjoint  equations  for  the  constrained  game 


B  =  j  ( v  sin  0  +  Vw2-v*  cos*  0  I  -  F  2 

*  *  '  V  CJ 


A, 


v  =  -  F  — 
o 


A.  =  -  A 


f~2  2  2 

v  cos  0  v  sin  0  +  yw  -v  cos  0 


0  0  i 


[~2~ 


2  2  o 

v  cos  0 


X  = 

V 


-A. 


sin  0 
‘0  i! 


P- 


2  2  ou 

V  COS  0 


v  sin  0 


0  +  Jv/2- 


2  2 

v  cos  0  - v 


+  F 


3 

v  a 


Finally,  comparing  this  Hamiltonian  with  that  of  the  three  dimensional 
game,  we  obtain 


A0  /  2  2 
JJ  \'W  “V 


cos  0  =  wp  -  vAr  cos 


/  ^2 

(.2  ?>0 


which  can  be  rearranged  ,^o  as  to  yield  a  perfect  square  that  gives 


«  0  v  cos  0 

Ar  =  i 


vjw2-v2  cos2  0* 


0  w 

P  = 


'/w^-V2  cos2  0 
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Wo  recognize  the  relations  we  had  along  2).  But  here  p  is  no  longer 
arbitrary,  and  integration  of  the  adjoint  equations  is  needed  to  provide 
Ap  and  A^.  Their  initial  conditions  are  obtained  by  placing  (6.10)  in 
(6.7). 

The  constrained  trajectories  actually  lie  between  the  curve  of  tan- 
gency  points  and  2).  They  go  around  the  end  of  2)  in  a  state  constrained 
form  of  Isaacs'  .swerve  maneuver.  Due  to  the  fact  mentioned  earlier,  the 
resulting  two-dimensional  field  does  not  always  fill  the  empty  region  of 
the  capture  cylinder.  Figure  12  is  a  scale  drawing  of  the  capture  cylin¬ 
der  surface,  for  p  =  1. 

The  corresponding  three  dimensional  field  of  extremals,  if  it  does 
not  fill  the  whole  void  left  by  the  primaries,  still  gives  a  smooth  ex¬ 
tension  of  the  previous  field.  It  contains  trajectories  coming  from  the 
external  side  of  &. 

For  p  >  p.  ,  the  curve  B  closes,  and  this  construction  succeeds 
in  filling  the  portion  of  the  capture  region  non  accounted  for  by  the 
primaries  interrupted  at  fl .  Therefore,  it  gives  the  complete  solution 
of  the  game. 

6.4  Corner  Condition 

For  1  <  p  <  p  ,  we  expect  that  for  starting  points  with  small  v, 

“  z 

the  trajectories  of'  the  game  of  degree,  similar  to  those  of  the  game  of 
kind,  will  first  go  away  from  the  capture  cylinder,  and  then  reach  the 
field  already  known  and  follow  it.  Hence  the  need  for  a  corner  condition. 

Terminology.  We  are  interested  in  the  conditions  that  must  hold  on  a 
surface  S  where  two  fields  of  extremals  join.  S  locally  divides  the 
space  in  two  regions.  Let  the  incoming  trajectories  be  in  region  -,  and 
the  outgoing  one;?  in  region  +.  The  corresponding  sensitivity  vectors 

mm 

are  A  and  A  .  Let  n  be  the  normal  to  S  pointing  in  region  +, 


so  that  by  definition: 


FIGURE  12.  The  Constrained  Game 


i 

1 


(n,  f  (q>* ,  \Jr*)>  >  0 

A  simple  result  is  the  following  "jump  condition": 

Proposition 

_  4* 

A  =  A  +an  (6.11) 

Proof .  V(z)  is  assumed  to  be  uniquely  defined  in  the  whole  space.  From 
its  values  on  S,  its  directional  derivatives  in  a  plane  tangent 
to  S  are  uniquely  determined.  And  these  are  the  projection  of 
A  in  that  plane.  This  proves  the  proposition. 

Notice  that  the  argument  depends  on  the  existence  of  the 
directional  derivatives  of  V  in  S,  and  thus  on  the  existence 

of  an  open  neighborhood  of  z  in  S.  It  does  not  hold  as  such 
on  the  boundary  of  S. 

As  a  consequence  of  this  proposition,  once  S,  and  thus 
n,  is  known,  we  can  determine  A  ,  the  main  equation  for  re¬ 
gion  -  allowing  us  to  calculate  CC .  This  is  equivalent  to  saying 
that  we  can  solve  the  game  with  terminal  surface  S  and  terminal 

payoff  V(z) I  added  to  the  integral  part  of  the  performance  in- 
i  s 

dex.  We  still  need  a  condition  to  determine  S. 


Assumption.  Assume  that  in  a  neighborhood  of  S  we  have: 


<n,  f(<p~ ,\{r+)>  >  0 

<n,  f(cp+,f  )>  >  0 


(6.12) 


Namely,  none  of  the  players  can  prevent  the  state  from  crossing 
S  by  keeping  his  optimal  strategy  of  region-.  Then,  we  have  a 
result  similar  to  the  classical  corner  condition.  (See  [5]  and 
C93). 
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Theorem.  Under  condition  (6.12),  the  corner  condition  is 


X"  =  A+  (6.13) 

Proof .  Assume  one  of  the  players  decide  to  switch  earlier,  on  a  surface 
translated  from  S  by  Si.  Then,  because  of  (6.12)  the  other 
player  is  obliged  to  switch  also.  Taking  (6.11)  into  account, 
the  change  in  payoff,  to  first  order,  is 

5V  =  <a+-A),  Si>  =  -a<n,Si> 

If  Ci  is  not  zero,  this  quantity  has  the  sign  of  Ci,  and  thus 
there  exists  a  small  enough  bfi  such  that  the  variation  in  pay¬ 
off  has  that  sign.  Therefore,  the  player  whose  advantage  it  is 
should  have  switched  earlier.  This  ends  the  proof. 

At  this  point,  a  new,  typically  game-theoretic,  phenomenon  occurs: 
assume  that  one  of  the  two  inequalities  (6.12)  does  not  hold.  Then  the 
player  who  can  prevent  the  state  from  crossing  S  is  also  able  to  pre¬ 
vent  his  opponent  from  taking  advantage  of  the  potential  variation  of 
payoff  we  just  pointed  out.  Therefore,  a  switching  surface  S  can  occur 
with 


<n,  f«p+,'l")>  <  0  a  <  0  (6.14a) 

or 

<(n,  f(qT,^+)>  <0  Ci  >  o  (6.14b) 

Before  we  investigate  what  new  relation  replaces  (6.13),  wc-  make  the 
following  remark,  concerning  a  game  with  separated  dynamics: 

Proposition.  If  the  dynamics  are  separated,  one  at  most  of  the  two 
inequalities  (6.12)  can  be  false. 

Proof .  By  simple  calculation  we  have: 
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<n,  f(cp,\JO>  -  <(n,h(\Jr)>  -  <n,  g(cp)> 


by  definition  of  n,  we  have 

<n,f(<p~,f  )>  >0  < n,  h(\|r~)>  >  <n,  g(cp" )> 

<n,f(cp+,\[r+)>  >0  (n,  h0jr+))  >  <n,  g(cp+)> 

Assume  that 

<n,f(cp+,t-)>  <0  <n,h(\]T))  <  <n,  g(cp+)> 

together  with  the  previous  two  inequalities  yields 

<n,  f(cp  ,\(r+)>  =  <n,  h(\|f+))>  -  <n  g(cp  )>  >  0 
and  similarly  for  the  other  case.  This  proves  the  proposition. 

The  indifference  condition.  Assume,  for  definiteness,  that 

<n,  f(<p+,  ijf )>  <  0  (6.14a) 

Then,  according  to  what  we  said,  we  can  have,  on  S,  V.  <  0. 

If  upon  reaching  S  the  pursuer  changes  his  control  but  the  evader 
does  not,  the  state  drifts  back  into  region  The  pursuer  must  switch 
back  to  cp  ,  causing  the  state  to  reach  S  again.  The  sequence  is 
then  repeated,  inducing  "chatter".  We  assume  a  convex  vectorgram  so  that 
chatter  can  always  be  replaced  by,  and  cannot  be  better  than,  a  simple 
strategy.  Thus  the  pursuer  must  choose  a  control  $  such  that  the 
state  follows  S: 


<n,  f (<p,'|0>  =  0 

we  call  (<p,\Jr  )  the  traversing  strategies  and  (<p  ,\{r  )  the  penetrating 
strategies.  S  can  be  the  switching  surface  of  the  optimal  game  only  if 
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the  traversing  strategies  still  optimal,  namely  if,  as  on  an  optimal 
trajectory,  the  rate  of  decrease  of  V  is  equal  to  the  value  of  the 
Lagrangian  with  the  corresponding  controls.  Notice  that  because  of  (6.11) 

<A“,  f(cp,'f)>  =  <  A+,f  (9,  f  )> 

so  that  the  Hamiltonian  is  uniquely  defined  and  our  condition  can  be 
written 


=  L(9,1{0  +  <A",  f (?,♦“»  =  0 

But  we  know  that 

min  H(A  ,9,ty  )  =  H(X  ,9  ,i{r  )  =  0  . 

cp 

We  therefore  have  this  complement  to  the  previous  theorem. 

Theorem:  (The  Indifference  Condition).  If  condition  (6.12)  does  not 

hold,  but  instead  (6.14a)  for  instance,  then  at  a  corner  point  we 
must  have  O'  <  0  and  in  addition  one  of  the  following  two  condi¬ 
tions  must  hold. 

.  <P  =  <P~ 


or 

•  Arg  minH(A  ,9,1?  )  non-unique. 

<P 

The  first  possibility  corresponds  to  the  switch  envelope,  a  phenom¬ 
enon  first  discovered  by  J.  V.  Breakwell  and  A.  W.  Merz  in  Isaacs’  homi¬ 
cidal  chauffeur  game  [20].  The  trajectories  reach  S  tangentially,  and 
then,  to  the  choice  of  the  evader  (or  of  the  pursuer  for  the  case  (6.14b)) 
the  state  can  either  follow  the  switch  envelope,  or  leave  it  on  a  trajec¬ 
tory  of  region  +. 

The  second  possibility  has  an  interesting  interpretation  when  the 
Lagrangian  is  independent  of  9,  in  the  minimum  time  problem  for  instance. 
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Then  P  (Q  in  the  case  (6.14b))  must  have  an  affine  set  as  part  of 
its  boundary,  and  A  must  be  normal  to  it.  In  particular,  if  P 
lies  in  an  affine  set  J  but  has  only  relative  extremal  points  in  -its 
boundary,  (a  line  segment,  a  conic  section)  then  A  must  be  normal  to 

y. 

For  a  two  dimensional  game,  the  only  proper  subspace  is  a  straight 
line,  so  that  the  possibility  we  have  just  discussed  can  occur  only  if 
one  of  the  players  has  a  linear  vectogram,  and  we  recognize  Isaacs’ 
equivocal  phenomenon.  We  see  that  in  the  higher  dimensions,  this  is 
not  required.  In  the  isotropic  rocket  game,  for  instance,  an  equivocal 
surface  of  this  type  could  occur  with  A  parallel  to  r*,  and  thus 
normal  to  the  plane  of  P. 

Let  us  finally  notice  that  our  description  (Chapter  5)  of  how 
two  semipermeable  surfaces  can  join  is  the  limiting  case  of  a  switch 
envelope.  The  unicity  assumption  we  had  to  make  merely  ruled  out  an 
equivocal  junction.  There  does  not  seem  to  be  any  reason  why  this  could 
not  occur.  We  did  not  describe  it  because  we  did  not  need  it,  and  be¬ 
cause  it  is  clear  how  the  present  more  general  theory  would  apply.  The 
case  (6.13)  leads,  for  barriers,  to  a  smooth  surface,  since  the  adjoint 
is  normal  to  it. 

6.5  The  Switch  Envelope 

i)  The  Indifference  Condition.  In  our  game,  we  already  know  a 
line  where  a  corner  occurs:  The  envelope  junction.  If  we  regard  the 
barriers  as  limits  of  the  trajectories  of  the  game  of  degree,  we  con¬ 
clude  that  we  must  find  a  switch  envelope  passing  through  that  line. 

The  choice  of  strategies,  traversing  or  penetrating,  would  rest  with 
the  evader,  and  a  be  negative. 

Let  the  switching  surface  bo  a  surface  X  given  as 
v  =  v(x,y) 

Then,  its  normal  is 
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bv 

3?  =  "p 

dv 

3?  =  -q 

i 

(We  have  used  the  traditional  notation  p  here,  not  to  be  confused  with 
the  parameter  of  our  game.  This  notation  appears  only  in  the  present 
subsection).  According  to  (6.11) 

A  =  A+  -  ap 
x  x 

A  =  A+  -  aq  (6.11a) 

y  y 

A  =  A+  +  a 

V  V 

and  we  have  two  relations.  The  main  equation  of  region  -  and  the  in¬ 
difference  condition  which,  here,  is  a  tangency  condition 

H(A  ,<p  )  =  -F0  +  wp  -  vAy  +  1  =  0 

-  <n,  f(cp  )>  =  -F  •  (py-qx)  (6.15) 

v  o 

+  £=|pA*  +  qA*  -  a(p2  +  q2)j  ~vq  +  F  “  -  0 


where 
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Conceptually,  the  parameter  a  can  be  eliminated  between  the  two  equa¬ 
tions  (6.15),  leaving  a  first  order  partial  differential  equation  on 
v (x , y) . 

Actually  it  can  be  seen  that  this  elimination  can  be  avoided,  by 
directly  finding  the  equations  of  the  characteristics  of  this  system  of 
equations.  If  the  two  relations  are  written 

H(x,y,v,p,q,a)  =  0 

G(x,y,v,p,q,a)  =  0 


define 


r„,  Bh  Bg  Bg  Bh 
cx]  =  3a  "  5^  35 

and  similarly  for  [Y] ,  [V] ,  [P]  and  [Q] ,  and  then  the  characteristics 
have  the  same  equations  as  for  a  single  partial  differential  equation: 

dx  dy _ dv  _  ~dp  -dq  _d. 

TpT  [Q]  p[PJ  +  q[Q]  [x]  +  ptV]  CY  J  +  q  [V] 

and  an  additional  relation  is  needed  to  propagate  a.  It  is  easy  to 
derive  as 

J/v  IVBh  Bg  Bg  Bh\  /Bh  Bg  Bg  Bh\  /Bh  Bg  _  Bll\ 

c,a  =  H  +  ^  SJ/  +  Pl^  S5  Bv  Ip) 

/Bh  Bg  Bg  BhY]  .. 

+  ql57  35  -  37  aq/Jds 

We  now  have  a  problem  of  Cauchy:  pass  an  integral  of  (6.15)  through  -j  . 
It  is  known  that  unless  j  is  a  characteristic,  this  problem  has  a  well 
defined  solution. 

ii)  Shape  of  X .  Two  major  difficulties  occur  in  trying  to  actually 
compute  an  integral  of  (6.15). 
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First,  on  A  is  infinite,  so  that  only  its  direction  is  known. 

This  is,  in  principle,  compensated  for  by  the  fact  that  we  have  another 
piece  of  information:  we  know  the  direction  of  A  ,  namely  v  .  Thus 
we  can  choose  the  direction  of  n,  normal  to  in  such  a  way  that  the 
limit  of  A  for  points  tending  to  'J  be  v  .  But  this  program  is 
exceedingly  difficult  to  carry  out.  It  would  have  to  be  done  by  itera¬ 
tion  on  the  position  of  n  as  a  function  of  the  point  of  j  considered. 

In  addition,  the  problem  is  made  even  more  difficult  by  the  fact 
that  A  is  not  known  explicitly  as  a  function  of  the  state.  It  is 
computed  by  completely  separate  means,  together  with  the  state  along  a 
trajectory.  Thus,  what  is  known  numerically  only,  is  a  family  of  func¬ 
tions  z(t  ,  t  ,  x  ),  A(t  ,  t.,  ,  t  )  where  t  is  the  tangency  t,  t  and 
t2  are  the  parameters  along  the  state  constrained  and  the  unconstrained 
trajectories.  A  computation  in  the  (r  , r , t )  space  makes  the  quan- 

C  -L  £• 

titles  p  and  q  become  complicated  functions  of  the  variables  that  we 
cannot  express  explicitly  either. 

For  these  reasons,  numerical  integration  of  our  partial  differential 
equation  did  not  appear  to  be  feasible  within  the  scope  of  this  disserta¬ 
tion.  But  some  interesting  results  can  be  found  about  the  shape  of  a 
solution. 

The  way  the  problem  of  Cauchy  is  solved  is  the  following:  at  each 
point  of  $  the  partial  differential  equation  d efines  a  cone  of  possible 
normals  to  the  surface.  The  requirement  that  it  be  normal  to  j  deter¬ 
mines  this  normal.  Then  the  necessary  initial  conditions  are  known  to 
integrate  the  equations  of  the  characteristics.  At  the  end  J  of 
the  second  requirement  disappears,  and  all  the  directions  satisfying 
the  equations  can  be  used.  Consequently,  the  solution  of  (6.15)  has  the 
desired  shape,  providing  a  field  of  extremals  similar  to  the  singular  sur¬ 
face  for  those  arriving  in  the  vicinity  of  J. 

We  can  also  show  that  this  solution  actually  lies  in  the  region  wo 
conjecture,  between  the  envelope  barrier  and  the  roof.  Equations  (6.11a) 
are  identically  satisfied  along  a  solution.  They  express  the  fact  that 
the  projections  of  A+  and  A  on  the  tangent  plane  to  X  are  equal. 

4.  — 

Therefore,  by  continuity,  the  projections  of  i  and  v  on  this  plane 
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at  'J  must  coincide.  The  vectors  v  being  normal  to  the  two  barriers, 
this  is  possible  only  if  the  tangent  plane  to  X  lies  in  the  dihedron 
defining  the  capture  region. 

Finally,  notice  also  that  the  directions  of  the  vectors  v  imply, 
as  must  be,  a  <  0. 

Figure  13  is  a  schematical  cross  section  of  the  barriers  by  a  plane 
v  =  constant . 


A- 


FIGURE  13.  Position  of  the  Switch  Envelope 

Because  wo  have  not  been  able  to  compute  this  surface — and  conse¬ 
quently  not  the  field — either,  we  cannot  carry  further  the  solution  of 
this  game. 

Some  questions  arise  about  the  possibility  that  the  construction 
proposed  is  part  of  the  solution  of  the  game  of  degree.  We  shall  try 
to  answer  them  and  to  present  a  reasonable  sot  of  conjectures. 

6.6  Conjectures 

A  first  question  arises  about  the  trajectories  of  the  (y,v)  plane. 
There,  we  know  that  the  optimal  trajectories  are  a  sot  of  parabolas, 
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deduced  from  each  other  by  translation  parallel  to  the  y-axis.  But  this 
poses  two  problems:  These  trajectories  belong  to  the  field  of  primaries, 
and  we  now  pretend  that  they  are  imbedded  in  another  field,  making  a 
corner  with  the  first  one.  And  furthermore,  this  corner  occurs  on  a 
switch  envelope,  so  that  these  parabolas  should  be  tangent  to  X.  But 
it  is  obvious  they  have  no  envelope,  so  that  they  cannot  be  tangent  to 
K,  unless  this  surface  itself  is  tangent  to  the  symmetry  plane.  This 
last  possibility  is  ruled  out  by  the  fact  that  belongs  to  X,  and, 
except  for  p  =  1,  is  not  tangent  to  the  symmetry  plane. 

We  conjecture  the  following  answer  to  these  questions.  The  switch 
envelope  surface  X  intersects  the  (y,v)  plane  at  a  nonzero  angle, 
and  not  jr/2  either,  along  a  line  £,  passing  through  A'.  The  trajec¬ 
tories  of  the  field  -  arriving  tangent  to  X  at  £  come  from  the  region 
x  >  0,  as  the  trajectories  of  the  envelope  roof  indicate,  thus  leaving 
between  them  and  the  symmetry  plane  a  region  not  accounted  for  by  this 
field. 

Along  £,  the  field  of  incoming  trajectories  is  interrupted.  In 
other  words,  due  to  the  discontinuity  of  the  normal  of  X,  the  restric¬ 
tion  of  V(z)  to  X  is  differentiable  in  a  closed  half  surface  only. 
Thus,  the  argument  that  the  component  of  X  in  the  tangent  plane  to  X 
is  continuous  does  not  hold  any  longer.  What  does  hold  is  that  the  com¬ 
ponent  tangent  to  £  is  continuous.  This  gives  one  less  condition. 

On  the  other  hand,  for  a  trajectory  arriving  at  £  the  evader  can¬ 
not  prevent  the  state  from  crossing  X,  again  because  of  the  angle  in 
that  surface  at  that  point.  Therefore,  the  requirement  that  this  trajec¬ 
tory  be  tangent  to  X  does  not  persist  either.  Therefore,  using  the 
degree  of  freedom  left  in  detei'mining  X  ,  we  can,  from  each  point  of 
£,  generate  a  one-parameter  family  of  extremals  extending  from  the  one 
tangent  to  X  to  the  parabola.  This  fills  the  void  left  by  the  previous 
field.  This  singular  roof  now  appears  as  the  natural  limit  of  this  con¬ 
struction,  giving  strength  to  the  conjecture. 

Notice  that  the  fact  that  X  -X+  must  be  normal  to  £  and  that 
X"  must  generate  the  parabolas  ns  optimal  trajectories  still  does  not 
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allow  to  compute  £  independently  of  X ,  as  any  adjoint  A  contained 
in  the  (y,v)  plane  generates  the  same  trajectories.  But  we  conjecture 
that  this  line  lies  in  the  region  y  <  vt^,  Q(t^)  =  0,  so  that  the 
field  of  primaries  is  interrupted  between  its  singularity  and  capture. 
The  parabolas,  then  go  through  Q,  but  are  imbedded  in  a  field  with  no 
singularity  there. 

We  shall  now  present  a  conjecture  giving  a  tentative  solution  of 
the  problem  of  the  termination  of  the  singular  carrier,  left  unsolved 
in  the  previous  chapter.  We  have  seen  that  there  is  a  point  B’  where 
a  trajectory  of  the  singular  surface  4  is  tangent  to  the  capture  cyl¬ 
inder.  Extending  from  B'  into  the  capture  region,  there  is  a  locus 
of  points  where  the  trajectories  incoming  to  X  are  tangent  to  C. 

From  each  of  these  points,  we  can  generate  retrogressively  a  constrained 
trajectory,  and  incoming  to  these  a  three  dimensional  field  of  tribu¬ 
taries.  Qualitatively,  the  constrained  trajectories  look  like  2)’  and 
the  tributaries  like  the  trajectories  of  §' . 

Let  C  be  the  point  wl'iere  the  trajectory  of  4  through  B'  is 

i 

tangent  to  the  cusp  on  4  •  From  this  point,  the  envelope  barrier  g' 
cuts  the  trajectories  of  X-  Similarly,  the  field  of  tributaries  just 
described  will  interfer  witlj  the  field  of  trajectories  incoming  to  X» 
Thus,  a  dispersal  surface  will  occur,  a  locus  of  points  where  the  time 
to  go  in  both  fields  is  equal. 

The  trajectory  of  4  through  B'  belongs  to  both  fields,  and  C 
will  thus  belong  to  the  dispersal  surface..  Therefore,  this  surface 
intersects  4,  and,  as  the  rest  of  its  field,  4  must  be  interrupted 
along  that  line. 

Here  again,  V{z)  must  be  uniquely  defined  on  the  dispersal  sur¬ 
face.  The  only  way  in  which  the  field  on  one  side  can  have  an  infinite 
gradient  and  not  the  field  on  the  other  side,  is  for  the  dispersal  sur¬ 
face  to  be  tangent  to  the  barrier  at  their  intersection.  Figure  14 
sketches  this  situation. 
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FIGURE  14. 


Singular  Barrier  and  Dispersal  Surface 


Since  following  &  does  not  enable  the  evader  to  avoid  capture, 
there  is  no  contradiction  in  having  an  optimal  trajectory  of  the  game 
of  degree  have  a  common  point  with  the  barrier.  This  is  similar  to  what 
happens  in  the  Homicidal  Chauffeur  [20]  and  other  games. 

This  construction  still  does  not  fill  the  whole  state  space  with 
extremals.  Another  field  would  join  on  the  one  just  constructed, 
possibly  through  an  equivocal  surface  generated  from  the  end  of  the 
singular  barrier.  The  common  trajectory  to  &  and  §',  through  B' 
and  C,  also  would  have  to  be  considered  as  it  is  the  end  of  the 
barrier  £’ ,  and  the  other  edge  of  the  "hole"  left  in  the  proposed 
barrier. 

6.7  Conclusion 

Our  investigation  of  the  Isotropic  Rocket  Game  can  be  summarized 
as  follows: 

The  linear  theory  shows  that  for  p  <  1,  capture  always  occurs, 
in  a  line  not  larger  than  an  estimating  function  we  were  able  to  con¬ 
struct  explicitly.  However,  this  time  is  not  always  optimal. 

A  more  classical  app3.’oach  led  to  the  following  situation: 

p  <  1.  We  exhibited  a  composite  field  of  extremals  made  of  part  of  the 
primaries,  a  safe  contact,  and  its  tributaries.  Although  no 
analytic  proof  is  available,  our  computations  indicate  that 
this  field  probably  fills  the  space,  with  no  conjugate  point, 
and  thus  provides  the  complete  solution  to  the  game. 
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1  <  p  <  .  We  have  been  stopped  in  our  investigation  by  an  excessive 

technical  difficulty  in  the  integration  of  a  partial  dif¬ 
ferential  equation.  Since  any  sufficiency  argument  rests 
upon  the  construction  of  a  complete  field  of  extremals,  we 
can  only  propose  conjectures.  We  have  not  been  able  to 
display  a  closed  set  of  barriers.  Furthermore,  we  have 
proposed  a  construction  that  would  give  trajectories  going 
around  the  known  barriers.  Consequently,  our  tentative 
conclusion  is  that  capture  occurs  from  any  initial  condi¬ 
tion.  For  outer  regions  of  the  state  space,  an  optimal 
chase  would  then  involve  eight  stages  or  more. 

p  <  p  <  p  .  We  have  found  a  set  of  semipermeable  surfaces  forming  a 

JL  '  Z 

closed  barrier:  it  defines  a  closed  capture  region.  If 
the  game  starts  outside  this  region  capture  will  never 
occur.  If  it  starts  inside,  capture  will  always  occur. 

For  a  small  portion  of  this  region,  the  optimal  trajectories 
seem  to  involve  the  same  singularity  as  in  the  case  1  <  p  < 
p  ,  that  we  have  not  been  able  to  compute. 

p  <  P.  We  still  have  a  closed  barrier,  of  simpler  configuration 

2 

than  in  the  previous  case.  The  same  field  as  in  the  case 
p  <  1  seems  to  account  for  the  whole  capture  region,  again 
providing  the  complete  solution. 

Conclusions 

Information  Structures .  We  have  seen  that  Pontryagin's  direct 
method  can  be  extended  to  various  information  structures.  In  the  basic 
form,  the  pursuer  knows  the  evader's  control  for  a  time  e  in  the  future 
Letting  6  grow  to  infinity  gives  the  case  where  the  whole  future  con¬ 
trol  of  the  evader  is  known.  Giving  a  fixed  finite  value  to  e  leads 
to  a  discrete  estimating  function  T(z,e).  It  is  possible,  in  partic¬ 
ular,  to  check  directly  that  for  a  given  point  z,  T(z,e)  increases 
as  e  is  decreased.  Pontryagin  proposes  a  construction,  the  alternating 
integral,  that  gives  an  estimating  function  valid  for  every  positive  e. 
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This  suggests  an  investigation  of  the  limit  of  the  pursuit  process  pro¬ 
posed  as  €  goes  to  zero. 

We  modify  somewhat  Pontryagin's  process  by  considering  the  minorant 
game  or  lower  e-strategy  where  the  players  choose  their  controls  sequen¬ 
tially.  Then,  using  results  of  Fleming  and  of  Friedman  on  the  convergence 
of  the  game  as  e  goes  to  zero,  we  have  a  well-defined  notion  of  contin¬ 
uous  strategy  that  lends  itself  to  the  following  analysis.  We  have  given 
an  explicity  characterization  of  the  strategies  obtained  by  this  varia¬ 
tion  of  Pontryagin's  technique.  This  allowed  us  to  show  that  they  have 
a  well  defined  limit  as  e  goes  to  zero,  that  we  were  able  to  characterize 
in  a  way  reminiscent  of  the  Maximum  Principle.  An  interesting  result  is 
that  for  a  wide  variety  of  games,  this  limit  strategy  of  the  pursuer  is 
independent  of  the  control  chosen  by  the  evader  at  current  time. 

Optimality.  Since  in  the  unsymmetric  approach  taken,  the  pursuer  has 
some  knowledge  of  the  future  control  of  his  opponent,  it  is  possible 
to  seek  a  pursuit  strategy  optimal  against  that  control,  which  is  more 
than  usual  saddle  point  condition.  The  strategy  proposed  by  Pontryagin 
achieves  this  goal  provided  that  the  estimating  function  is  the  optimal 
time  to  go  in  the  usual  sense.  We  therefore  investigated  the  optimality 
of  this  pursuit  time.  We  found  a  condition  under  which  there  is  an  e 
small  enough  so  that  this  time  is  locally  optimal  with  the  corresponding 
e-strategy.  Then,  using  our  definition  of  a  continuous  strategy,  we 
proved  that  for  that  limit,  the  estimating  function  always  has  this  local 
property.  Some  of  the  intermediate  results  derived  for  these  two  proofs 
are  of  interest  by  themselves. 

We  have  described  the. phenomenon  by  which  the  global  trajectories 
can  still  fail  to  be  optimal.  Understanding  this  mechanism  gives  us 
direct  means  of  checking  the  optimality  of  the  trajectories  computed  in 
any  instance.  We  also  derived  a  condition  under  which  we  are  assured  of 
this  optimality.  Unfortunately,  it  is  not  explicit  and  not  much  simpler 
than  the  direct  check.  However,  it  allowed  us  to  derive  several  more 
restrictive  sufficient  conditions,  including  an  eax*lier  one  by  Gusyatnikov 
and  Nikolsky. 
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Nonregular  Points.  The  previous  analysis  emphasized  the  importance  of 
Pshenichnyi ' s  nonregular  points.  We  proved  that  they  are  points  where 
the  gradient  of  the  estimating  function  is  infinite.  We  distinguished 
between  two  kinds  of  such  points.  The  first  kind  was  shown  to  be  associated 
with  Isaacs'  concept  of  a  barrier.  The  second  kind  is  not  as  well  under¬ 
stood.  An  explicit  example  was  nevertheless  given  to  show  that  it  can  occur. 

Multistage  Games.  The  construction  proposed  by  Pontryagin  naturally 
leads  to  the  consideration  of  multistage  games.  It  was  seen  that  most 
of  the  concepts  introduced  in  the  continuous  case  carry  over  to  the  dis¬ 
crete  case.  Our  main  interest  has  been  in  a  slight  modification  of  the 
resulting  discrete  theory:  the  system- theoretic  formulation  where  the 
controls  are  unbounded  and  capture  is  defined  as  point  coincidence  modulo 
a  subspace.  In  that  case,  the  constructions  performed  take  a  very  simple 
form,  and  explicit  criteria  were  given  on  the  coefficients  of  the  matrices 
involved  for  capture  to  be  possible  with  each  of  the  three  main  informa¬ 
tion  structures.  We  also  considered  the  case  where  the  capture  subspace 
is  invariant  under  the  free  dynamics  of  the  system,  and  generalized  to 
the  vector  valued  controls  case  an  earlier  strong  controllability  theorem 
by  Kalman. 

Isotropic  Rocket  Game.  In  the  second  part,  we  investigated  an  example 
considered  by  Isaacs:  the  Isotropic  Rocket  Game.  An  object  having  an 
acceleration  of  constant  magnitude  tries  to  get  within  a  distance  i  of 
an  object  having  a  velocity  of  constant  magnitude.  We  generalized  Isaacs' 
formulation  by  assuming  that  the  chase  occurs  in  the  three  dimensional 
physical  space,  but  immediately  proved  that  an  optimal  chase  occurs  in 
a  fixed  plane,  and  has,  in  a  suitable  set  of  reference  axes,  linear  dyna¬ 
mics.  The  state  space  is  then  four  dimensional,  as  the  components  of  the 
pursuer's  velocity  must  be  considered  together  with  the  relative  coordin¬ 
ates  of  the  evader  with  respect  to  the  pursuer.  The  linear  theory  was 
applied  to  that  representation.  It  shoved,  in  particular,  that  when  the 
unique  parameter  p  of  the  game  is  smal.'er  than  one,  capture  occurs 
from  all  initial  conditions.  This  is  a  good  approximation  of  the  com¬ 
plete  situation  in  that  the  limiting  valus  of  the  parameter  we  found  is 
very  close  to  one.  However,  this  game  exhibits  the  typical  phenomenon 
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where  the  linear  theory  fails  to  yield  the  optimal  solution.  Therefore 
we  could  not  conclude  from  that  theory  that  for  p  >  1  the  evader  would 
escape,  and  we  found  later  that  this  is  probably  not  true. 

Game  of  Kind.  We  used  a  reduced  three  dimensional  state  space  to  pursue 
the  analysis  of  this  game  using  Isaacs'  generalization  of  the  Hamilton 
Jacobi  theory.  We  first  investigated  the  game  of  kind.  We  emphasized 
the  concept  of  local  cone  of  semipermeable  directions,  and  gave  a  geometri 
cal  construction  of  that  cone.  In  particular,  we  were  led  to  the  considera 
tion  of  a  new  type  of  semipermeable  surface  generated  from  one  point  by 
that  cone  of  directions:  the  singular  surface.  We  also  used  that  concept 
to  give  a  simple  description  of  Isaacs'  envelope  barrier  concept.  We  saw 
that  this  envelope  barrier  can  be  considered  as  a  particular  application 
of  a  more  general  result  as  to  how  two  semipermeable  surfaces  can  join  at 
a  non  zero  angle  in  a  barrier:  the  envelope  junction. 

Game  of  Degree.  We  finally  investigated  the  game  of  degree,  for  which 
we  proposed  a  derivation  of  the  genei ilized  Hamilton- Jacobi  equation  by 
Caratheodory's  technique.  The  game  of  degree  involves  a  safe  contact 
first  pointed  out  by  Breakwell  and  Boardman.  It  is  similar  to  Isaacs' 
envelope  barrier.  We  introduced  the  concept  of  singular  state  constraint 
and  proved  that  if  a  state  constraint  is  not  singular,  optimal  trajec¬ 
tories  reach  and  leave  it  tangentially.  This  result  can  be  shown  to 
imply  a  result  by  Weierstrass  about  the  same  question.  We  also  derived 
a  general  corner  condition  for  differential  games.  It  was  found  that  it 
can  take  two  forms:  either  the  same  requirement  as  in  the  one  player  op¬ 
timization  that  the  adjoint  vector  be  continuous,  or,  if  a  certain  in¬ 
equality  is  not  satisfied,  the  indifference  condition.  This  condition 
itself  can  take  two  forms,  one  having  Isaacs'  equivocal  phenomenon  as 
a  particular  case,  the  other  one  being  Breakwell 's  switch  envelope. 

The  detailed  solutions  for  the  Isotropic  Rocket  Game  was  discussed 
at  the  end  of  Chapter  Six.  A  detailed  description  of  the  barrier  was 
given  at  the  end  of  Chapter  Five.  We  left  the  problem  unsolved  for  a 
small  range  of  the  parameter  p.  The  solution  in  this  interesting  case 


seems  to  be  very  complicated.  We  conjectured  that  it  involves  at  least 
an  eight-stage  chase,  with  two  corners  and  two  safe  contacts. 
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I  > 

\  l 
t  t 


&  1 


EQUATIONS  OF  THE  EXTREMALS 


We  start  with  the  simplest,  four-dimensional  description  of  the  game 


X  =  -  U  +  w 


Y  =  -  V  +  w 


U 

V 


Fx 


=  F_ 


(A.l) 


and  we  introduce  the  reduced  coordinates 


v  =  Vu2  +  V2 
X  s  -i  (VX  -  UY) 
y  =  -i  (UX  +  VY)  . 

We  define  the  angle  A  between  the  two  systems  by 


(A. 2) 


sin  «  =  — 


cos  A  =  — 


and  introduce  new  control  variables  cp  and  by 


F^  sin  A  +  Fy  cos  A  =  F  cos  9; 


w  sin  A  +  w  cos  A  =  w  cos  \Jr; 

A  i 


F..  cos  A  -  Fv  sin  A  =  F  sin  9  ; 

A  i 


w  cos  A  -  w  sin  A  =  F  sin  ; 
a  y 
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so  that  9  and  \jr  are  the  angles  defining  the  position  of  F  and  w  in 
the  new  coordinate  system,  all  angles  being  measured  clockwise  from  the 
y  axis. 

Differentiating  (A. 2)  and  placing  (A.l)  in  it,  and  taking  the  last 
transformation  into  account,  we  obtain  the  equations  of  the  game  in 
the  three-dimensional  representation 

•  V 

x  =  -  F—  sin  9  +  w  sin  \lr 

v 

y  =  F  —  sin  9  +  w  cos  -  v  (A. 3) 

v  =  F  cos  9  5 

and  we  want  the  equations  in  cylindrical  coordinates 

x  =  r  sin  0 

(A.4) 

y  =  r  cos  9  . 

We  differentiate  (a.4),  place  (A. 3)  in  it  and  solve  for  r  and  0  .  We 
obtain 


r  =  w  cos(\(f  -  0)  -  v  cos  0 

0  =  -  ^  sin,  9  +  —  sin(\|;  -  0)  +  ~  sin  0  (A. 5) 

V  =  F  cos  9  . 

Now,  we  need  the  equations  relating  the  various  representations 
of  vJ  =  X  [we  call  J  the  value  of  the  game  to  avoid  confusion  with  the 
coordinate  V  of  (A.l)].  Let  J  be  given  by 

J  =  JX(X,  Y,  U,  V)  =  J2(x,  y,  v). 


ldG 


Let 
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which  simply  checks  that  (X^,  X  )  and  (A^,  Ay)  are  the  same  vector 
\  expressed  in  their  respective  coordinate  systems.  By  the  same 
means,  we  compute  and 

.  .  (SS  *  v)  is  _  (ffi  .  x)  *  ♦  2  a 

U  V  V  V  v  v  v 

A  =  -  (25  .  x)  is  .  Vjr  .  y)  Xj;  +  v 

V  V  V  V  *  v  v  v 


from  which  it  follows  that 


\  -  ^“VvV 


(A,6a) 


which  means  the  dot  product  v  •  X^  is  conserved  in  the  change  of 
coordinates  .  We  obtain  also,  after  a  rather  lengthy  calculation 


2  A  9  2  1  9  r. 

°  =  =  h  <>*„  -  *v +  x; 

v 


while 


(A. 7) 


P2=  4  +  A?  =  Xx  +  Xy 


is  obvious  from  (A. 6). 


Finally,  we  make  a  purely  geometric  transformation  on  the  radius 
vector  r  and  on  a  perpendicular  axis 


rX  =  xX  +  yX 

r  x  y 

X  =  yX  -  xX 

0  x  y 


(A.  8) 


hence,  obtaining  other  forms  for  p  and  a 


p-  «  x;  +  -§ 

r 


0.2 

a-  =  — it  +  x 

“  V 

V 


(A , 8n ) 


This  justifies  the  notations  used  in  the  main  body  of  this  dissertation 

We  are  going  to  find  the  equations  of  the  extremals.  The  Euler- La grange 
equations  are  more  easily  integrated  in  the  four-dimensional  representation 
But  as  we  know  that  the  absolute  orientation  in  the  geometrical  space  is 
ignorable,  we  shall  always  assume  that  at  the  instant  of  capture,  t=0,  we 
have  A  =0,  or  U  =  0,  V  =  v.  Consequently,  the  expressions  for  A^  and 
A^r  which  we  have  are 


\ 

\  \ 
^  =  ^o 


V 


\ 


x 

v 


and  we  shall  use  these  relations  in  the  equations  of  the  trajectories 
because  our  ultimate  aim  is  to  have  their  equations  in  the  three  di¬ 
mensional  system,  as  a  function  of  the  parameters  x  ,  y  ,  v  ,  X,  ,  X  , 
’  o’  o'  o'  xo'  yo' 

at  r  =  0.  As  the  system  is  time  invariant,  the  same  equations  can 
be  used  from  any  such  set  of  "final”  conditions,  even  if  they  do  not 
correspond  to  capture.  In  particular,  we  have  used  them  to  compute 
trajectories  in  coming  to  the  various  singularities  in  the  game.  The 
Hamiltonian  is 


H  =  -Aj.U  -  AyV  +  AjA'j.  +  AyWy  +  +  AyFy  +  1 

and  its  maximized  form  is 

H*  =  -A^U  -  AyV  +  pw  -  FC  +  1  =  -F0  +  wp  -  vX  +1=0. 

The  adjoint  equations  are 


1-19 


0 


constant 


A., 


A..  =  0 


Ay  =  constant 


\ 


ay 


Au  =  -V+^o 

\  =  ~\X  +  XvQ 


= 


2c  t  +  a2 
o  o 


1 


where 


a  -  /  2  ,  ^  2 

=  \u  +  X$ 


:o  =  ^  +  XvoAY  ‘ 


A^  and  Ay  will  be  used  to  denote  the  final  value  of  X^  and  X^ 
dynamics,  with  the  optimal  controls,  are,  for  the  velocities 


do  _  s  .  .  r  -V > 

dT  -  -U  -  F—  -  F  0(T) 


dV 

dT 


=  -V  =  F 


\ 


-AyT  +  XVo 


o(x) 


Define  I  (t)  as 


I  (T)  -  I  -£ 


The  above  equations  give 


U  =  -F^Ij  -  ^0I0) 


V  =  -  XVoI0)  +  vc 


.  The 
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The  other  two  state  variables  are  then  given  by 

•i 


T  \ 

X  =  x  +  \  UcJT  -  —  wT 

P 


J 


A. 


Y  =  y  +  VdT - wT 


We  integrate  by  parts 


j 


T  fT 

UdT  =  TU  -  xU’dr 


and  similarly  for  V  we  find 


A. 


Y  =  Y0  -  .  +  VT  +  f(ayi2  -  . 


(A. 9) 


The  Ik's  can  be  expressed  analytically.  Clearly,  we  have 


■J 


AL 


=  -  log 


Vp2^2  -  c  J  +  oj  9 


pzr  +  pa  -  c 


Pao  -  Co 


Then  1^  is  obtained  from  the  remark  that 


p  h  “  Vo  b  "  Co  =  K<T) 


and  I2  is  obtained  from  the  fact  that 


fitscKs)]  =  2p2  - 3  <=„  +  ao  shr 


so  that 


x2  =  ~~2  £°T  +  coi1  -  a2o io3  . 

2p 
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I 


p 


r 

r 


I 


The  expressions  for  U  and  V  can  be  made  simpler  by  introducing,  in¬ 
stead  of  I 

o 


L  “  <XvA  '  “oV'o 

and  it  gives 

u  =  "  pz  tV(T>  +;V,(t)3 

v  =  -  ^  cv(t)  "  V(t)]  +  vo  • 


(A.  10) 


x,  y,  and  v  are  obtained  placing  (A. 9)  and  (A.10)in  (A.2),  with  the 
expressions  we  have  given  for  Iq,  1^,  and  I^.  The  adjoint  are  ob¬ 
tained  using  (A. 6) 


X*  -  $V.  +  > 

Xy  =  -i[Ayvo  -  FK( t)  ]  (A.  11) 


''o\]T  -  ^2  [C„K<T>  +  -  XVoAx!2lo]  +  XVoVo  )  . 


The  general  formulas  for  x,  y,  and  v  are  exceedingly  complicated.  We 
have  done  the  computation  using  the  intermediary  quantities  X,  Y,  U, 
and  V.  This  completes  determination  of  the  extremals  in  the  general 
case. 

However,  our  formulas  break  down  for  c  =  par  ,  because  I  is 
'  o  r  o  o 

no  longer  defined.  This  happens  when  the  vectors  (pQ,  XVq),  and 
are  parallel,  and  in  the  case  of  the  primaries  in  particular. 

Assume  we  have 
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(A. 12) 


Placing  this  in  the  equations  for 


Ay  and  Ay,  they  become 


\  -  V-t  +  7> 

av  -  y-”7> 

a  =  |cq  -  pr|  k(t)  a  JaQ  -  px|  -  oQ  » 

Then  the  velocities  are  given  by 


dU 

\ 

F  —  • 

\  -  pt 

dT 

P 

lXo  -  PTI 

dV 

\ 

F  —  - 

x„  -  PT 

dT  = 

p 

b* 

Q. 

1 

O 

X 

and  integrating,  we  find  a  formula  valid  for  all  T's 


A„ 

U  =  -F  ~  K(T) 

P 

A 

V  =  -  F  -§  K(t)  +  v  . 

p 


(A. 13) 


Then,  we  must  consider  the  integrals 


=  \  U(s) 


ds 


integrating  first  from  T  =  0  to  T  =  aj p,  then  for  T>  0Q/p 


T  ^ 


Av  fT 

x  =  F  ~2  \  (-Ps)d 

p  p  O' 


A*2 

s  *  *JT  » 
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c 

5 


$ 

? 

s 

5, 


«* 

t 

fe 


| 

l 


P 

T 

xp  =  "p^-  \  (Ps  ■  2°0)ds]  =  *^[p 

^o/p 

We  notice  that  these  two  expressions  can  be  written 

*P  =  F  i^[(T  -  ^)K(T)  ■  TOo] 

YP  ■  F^[(T-f)K(T)  '  Tffo]' 

Now,  if  we  take  the  corresponding  expressions  for  the  general  case, 
place  (A. 12)  in  them  assuming  that  I  is  defined,  we  find  identically 
the  same  equations.  Consequently,  one  can  use  the  same  formulas, 
checking  first  whether  cq  -  pa  A  0,  and  assigning  any  value,  say 
zero,  to  I  if  c  =  pa  . 

Finally,  since  it  gives  simple  formulas,  it  is  worth  deriving  the 

in  the  case  of  the  primaries 
our  formulas,  we  find 

Tv 

+  F-)t 

+  £1)t  +  v  x 
2  o 

Vo  • 


equations  for  the  reduced  coordinates 
where  XVq  =  =  0.  Placing  this  in 

y  / 

X  =  xo--(w 

K 

y  =  yQ  -  —  (* 
Ay 

U  =  -  F  —  T 

P 


A3, 

V  -  -  F  — —  x  + 

P 


_T 

2 


2a  t  + 
o 


as  a  single  formula 


voT  ‘ 


We  apply  to  this  the  transformation  (A. 2),  and  introduce  the  parameters 
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x  =  £  sin  ‘8 

o  K 

y  =  £  cos  8 

o 

v  =  s 

o 

and  the  intermediary  quantities 


Ax  =  p  sin  8 
Ay  =  p  cos  8 


§  =  s  sin  8 
q  =  s  cos  8  J 


we  obtain  for  the  three-dimensional  representation 


x  =  |  Q(T) 

y  =  Q(t)  +  VT 

v  =  U2  +(n  -  ft)2]* 


where  Q(  t)  =  FT2/ 2  -  wT  +  i. 
The  adjoints  are 


It  is  interesting  to  notice  some  simple  combinations 


(A. 15) 


\ 

0  =  pT 
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and  H  =  1  +  (w  -  rj)p  =  0,  yielding 

1 

p  =  -  . 

i]  -  w 

These  are  all  the  equations  we  need  in  the  last  two  chapters  for  the 
theory  as  well  as  for  numerical  computations. 


Appendix  B 


COMPUTATION  OF  $ 


We  want  to  compute  a  curve  $  on  the  surface  g  .  Let  us  review 
how  the  various  elements  of  the  problem  are  given. 

The  curve  ID  is  obtained  by  numerical  integration  of  Eq.  (5.8). 

We  consider  it  as  parametrized  by  the  value  s  of  the  velocity.  Once 
a  point  of  D  is  known,  we  have  analytical  formulas  for  the  trajectory 
of  g  tangent  at  that  point  and  for  the  adjoints  along  it,  as  a  func¬ 
tion  of  the  time  to  go  T,  from  the  point  of  g  to  2).  Therefore,  a 
point  of  g  is  represented  by  its  parameters  (s,  T  ).  $  is  defined  by 

an  initial  point  and  a  field  of  directions  on  g.  This  direction  is 
obtained,  at  any  given  point  of  g,  by  solving  numerically  Eq.  (5.11a) 
for  y  .  This  angle  allows  us  to  compute  v  with  =  0.  This  vector 

can  be  placed  in  the  dynamics  to  obtain  the  direction  f(v, z). 

We  cannot  simply  use  an  integration  routine  with  that  direction  f 
for  two  reasons:  first,  due  to  round-off  and  truncation  errors,  the 
curve  would  quickly  drift  off  the  surface  g.  Even  if  this  was  avoided, 
it  would  not  give  the  parameters  of  the  computed  point  on  g.  As  we 
need  these  parameters  to  compute  vb  and  subsequently  y,  we  would  have 
to  do  some  extremely  tedious  interpolation  to  find  them. 

For  this  reason,  we  directly  look  for  .2  in  the  form  =  T^(s). 
Using  s  as  the  independent  variable  is  convenient  because  the  incre¬ 
ment  on  s  is  then  chosen  arbitrariJ.y,  and  (5.8)  integrated  once  up 
to  the  new  value.  The  rest  of  the  integrating  procedure  is  done  by 
varying  T  ,  using  analytical  expressions  to  obtain  the  corresponding 
point  z  =  (x,  y,  v). 

We  want  to  copy  a  classical  integration  routine  of  numerical 
analysis.  Such  techniques  use  the  quantity  dT^/ds  that  we  do  not 
have.  But  more  precisely,  what  is  really  needed  ore  such  expressions  as 
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(B.l) 


St. 


Ss 


where  several  such  increments,  with  various  arguments  of  the  function 
dT^/ds,  are  averaged  in  a  suitable  way.  In  our  case,  instead  of  com¬ 
puting  dT^/ds,  we  compute  a  vector  v  normal  to  the  intersecting  sur¬ 
face,  thus  to  4  ,  ST^  is  obtained  by  computing  the  value  T°  +  8t^ 
of  T  where  the  trajectory  of  parameter  s°  +  8s  intersects  the  plane 
normal  to  v  through  z°.  This  is  done  by  solving  for  in 

V  ♦  Sz  =  0  (b.2) 

where 


oz  =  z(t^,  s°+  8s)  -  z°  . 

Since  wo  have  analytical  formulas  for  z(t  ),  we  can  differentiate 
the  dot  product  with  respect  to  T  ,  and  thus  solve  (B.2)  in  an  effi¬ 
cient  way  with  Newton’s  technique.  Notice  that  for  3  the  angle  between 
the  trajectory  of  6  and  the  tangent  plane  to  the  roof  is  large 
enough  so  that  this  technique  is  quite  accurate. 

In  our  case,  computing  v  is  a  rather  long operation.  We  have 
tried  to  keep  the  number  of  times  we  compute  it  as  low  as  possible. 

For  that  reason,  we  have  used  an  Adams  method  of  order  four,  started 
with  a  Runge  Kutta  method  of  order  four. 

We  give,  thereafter,  scale  drawings  of  the  crest  (computed  by  the 
same  program)  and  of  for  p  =  1  and  p  =  1.001. 
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IGUUE15.  Junction  [J  and  Crest  for  p  =  1  and  1.001 


Appendix  D 


REGULAR  STATE  CONSTRAINT  AND  A  RESULT  BY  WEIERSTRASS 


In  Section  6.3  we  defined  a  regular  state  constraint  and  show  that 
if  an  optimal  path  contains  such  an  arc,  it  is  reached  and  loft  tangen¬ 
tially.  This  result  obviously  applies  to  the  control  problem,  the 
one-player  game.  Similar  problems  were  studied  in  calculus  of  varia¬ 
tions,  and  it  is  interesting  to  relate  the  two  approaches. 

The  problem  of  joining  an  unconstrained  arc  with  a  constrained 
one  was  considered  by  Weierstrass,  and  appeared  in  his  lectures  as 
early  as  1879  at  least  [5].  The  problem  he  deals  with  is  his  paramet¬ 
rical  form  of  the  calculus  of  variations.  In  control  theory  language, 
it  can  be  stated  as  a  problem  in  a  two-dimensional  state  space  with  the 
simplest  dynamics 

(D.l) 

and  the  objective  is  to  minimize 

J  =  J  F(>v  -V  Ul>  U2)dt 

where  F  is  assumed  to  be  of  class  in  all  of  its  arguments  (for 

2  N 
our  purposes,  C  suffices). 

Weierstrass  wants  the  integral  to  depend  only  on  the  geometrical 
path  followed.  He  shows  that  the  necessary  and  sufficient  condition 
for  this  to  bo  true  is  that  F  be  homogeneous  of  degree  one  in  (u^,  U9). 

kUx,  kU2)  =  1^(-V  V  V’  U2^  k  >  °  ’ 


161 


and  a  consequence  of  this  is  that  there  exists  a  function 


such  that 


Fl(Xl>  V  V  V 


'Wl 


(D.2) 


where 


i  J 


=  u2f 
11 


i  =  1,2 
j  =  1*2  , 


We  add  that  the  second  partial  derivatives  of  a  homogeneous  function  of 
degree  one  are  homogeneous  of  degree  minus  onej  thus,  F^  is  homogeneous 
of  degree  minus  three  in  (u  ,  U  ). 

We  introduce  Weierstrass'  excess  function,  which,  because  of  the 
homogeneity  of  F  can  be  written  as 

®(*'^>X2,^l',^2,'*l,^2^  =  ^1  J^FUi^xi^ X2; ^1; ^2^  "*  FUi^xx,X2,^1,^2^  * 


U2[fu2(x1,x2^1^2)  " 


assuming  that  the  controls  are  the  cosine  and  the  sine  of  the  same 

o  2 

angle  y  (we  can  always  chose  U  +  U_  =  1  by  rescaling  and  using  the 

X  « 

homogeneity  of  F),  and  using  the  mean  value  theorem,  one  can  express 
E  by  means  of  F^  .  The  form  found  shows  that  if 
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x2’  C0S  r>  sin  =  0  VT 


then  the  relation  E  =  0  can  be  verified  only  if 


U1  =  U1 


U2  =U2 


which  shows  that  the  boundary  is  met  and  left  tangentially.  The  problem 
of  calculus  of  variations  is  said  to  be  regular  if  (D.3)  holds  at 
every  point.  Notice  that  from  the  homogeneity  of  F  ,  (D„3)  is  equiv¬ 


alent  to 


\(\>  V  U2)  £  0  if  (oj  +  U2)  £  0  .  (D.3a) 


We  shall  assume  that  the  coordinates  have  been  chosen  in  such  a 
way  that  the  boundary  of  the  state  space  is  x0  =  0  .  We  want  to 

Ct 

compare  the  condition  (D.3)  with  ours,  which  can  be  expressed,  in  this 
case,  as  follows:  Let  the  optimal  control,  obtained  by  minimizing 
the  Hamiltonian,  be  given  by 


U1  =  VX1>  V  V  X2) 

U2  =  U2^X1>  X2>  Xl>  X2^ 


and  we  ask  that  the  equation 


u2  =  0 


solvable  for  X2,  or  in  other  words,  that  there  exist  a  function  A0 


such  that 


U2[X1>  X2;  Xl>  V'V  X2;  Xl)]  =  0 


(D.4) 


We  form  the  Hamiltonian 
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H 


-  Vi +  V2  +  p(li> 


x 


2> 


As  and  are  unrestricted,  the  minimum  must  occur  at  an  interior 
point,  thus  given  by 


^  =  X1  +  FUi  =  0 

(D.5) 

^  =  X2  +  FU2  =  0  . 


Moreover,  since  we  want  a  minimum,  and  not  a  maximum,  we  must  have 


U2F,  >  0 
J  1 


i  =  1,2 
j  =  1,2  * 


(D.5a) 


Equation  (D.5)  must  be  used  to  determine  U  and  U  ,  (D.5a)  allowing 
to  choose  between  two  solutions.  First,  notice  that  multiplying  the 
first  equation  in  (D.5)  by  U^,  the  second  by  and  adding,  and  then 
making  use  of  Euler's  identity  for  homogeneous  functions,  we  find 


X1U1+  X2U2+  VUi  +  U2FU2  =  X1U1  +  X2U2  +  F  =  0 


showing  that  a  consequence  of  (D.5)  is  H  =  0.  This  is  not  necessarily 
possible  for  any  pair  (X^,  X9). 

Let  us  differentiate  (D.5)  at  a  fixed  point  (x^,  x^  using  (D.2) 


U2FldUl  '  WldU2  *  -dXl 

-UlU2FldUl  +  UXdU2  "  -dX2  ’ 


(D.6) 


The  first  consequence  is  that  this  system  can  be  satisfied  only  if 


VX1  +  U2dX2  "  0 


(D.6a) 
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which  shows  that  U  and  U  are  defined  only  as  long  as  X  and  X 

l  Zt  12 

satisfy  a  given  relation.  Equation  D.6a,  together  with  (d.5)  gives 
dH  =  0  and  is  consistent  with  our  previous  remark  that  H  has  to  take 
a  fixed  value,  namely  zero. 

We  assume  thus  that  X^  and  Xg  are  varied  simultaneously  according 
to  (D,6a),  Then  the  two  equations  (D.6)  are  redundant.  But  we  can 
arbitrarily  decide  to  choose  U  and  U  such  that 

J.  Za 

uj  +  uj  =  1  (D.7) 

since  their  ratio,  giving  the  direction  (U  ,  U  ),  only  matters.  Then, 

1  2 

we  have 

UldUl  *  U2dU2  =  °  (D.7a) 


and  placing  that  in  (d.6),  we  find,  making  use  of  (D.7) 


(D.8) 


where  U2  is  now  a  function  of  x^,  x^,  and  X9. 

This  relation  could  have  been  obtained  starting  from  the  beginning 


with 


x^  =  cos  y 

x2  =  sin  r 


(D. la) 


instead  of  (D.l),  and  having  y  as  only  control.  But  this  approach 
yields  less  information.  It  does  not  clearly  show  the  existence  of  a 
contraint  or  the  X's,  and  the  equality  H  =  0  must  be  borrowed  from 
the  general  theory  to  establish  (D.8). 

We  see  that  under  condition  (D,3),  U2  is  continuous  in  X0.  More¬ 
over,  from  our  assumptions  on  the  regularity  of  F,  under  condition 
(d.7)  is  bounded  from  above.  Therefore,  in  view  of  (D.5a),  we 
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have,  for  some  fixed  m 


<  m  <  0  . 


(D.9) 


This  still  does  not  prove  that  U2  =  0  has  a  solution,  because 
the  added  condition  on  the  X's  can  restrict  the  possible  values  of 
X  to  a  finite  interval,  or  to  a  semi- inf inite  one,  such  as  X^  <,  X° 

(or  X9  ^  X°). 

Assume  that  U9  is  defined  for  some  (X^,  X^).  Vary  X^  by  dX2, 

and  we  see  that  we  can  find  the  corresponding  variation  dX,  from  (D.6a), 

except  when  U  =  0.  In  that  case,  from  (D.7),  U9  =  1.  Assume  that 

the  allowable  X9's  have  an  upper  bound  of  X2  .  Then,  as  X2  increases 
o  o 

toward  X  ,  U2  decreases  toward  =  ±  1.  And  as  Ug  cannot  be  larger 

«  W  Q 

tlian  one,  we  necessarily  have  u  =  -1.  Similarly,  if  X  has  a  lower  bound, 
it  corresponds  to  U  =  +1.  This,  together  with  the  continuity  of  U 
and  with  (D.9),  shows  that  the  equation  U2  =  0  always  has  a  (single) 
root  in  X9 •  Therefore,  our  condition  (D.4)  is  satisfied. 

The  conclusion  is  that  Weierstrass1  condition  implies  ours,  which 
is  thus  more  general.  However,  it  must  be  noticed  that  the  earlier  is 
much  more  elegant  than  the  latter  in  that  it  can  be  readily  checked 
from  the  data  of  the  problem,  without  actually  determining  Ug(x  ,x2,Xg). 

Similar  simple  conditions  might  exist  for  the  control  problem  but 
we  did  not  investigate  this  question.  However,  it  is  interesting  to 
notice  a  typical  case  where  our  condition  is  not  met:  when  a  control 
determined  by  the  state  constraint  otherwise  has  a  "bang-bang"  behavior 
(and  except  if  the  constraint  happens  to  imply  an  extreme  value  of  the 
control).  A  typical  example  is  the  Dolichobrachistochrone  problem. 

In  these  cases,  the  additional  adjoint  is  actually  provided  by 
singular  arc  conditions.  This  shows  the  close  relation  between  singular 
solutions  and  the  question  of  how  a  state  constraint  is  joined  on. 
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